Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53377

pipeline zip step fails with absolute path

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      It appears that the zip step prepends an absolute path to the zip file name when trying to upload, resulting in an error if you passed in an absolute path.

      In scripted jenkinsfile:

      zip zipFile: "${WORKSPACE}/my.zip", archive: true, dir: "${WORKSPACE}/some/dir"
      

      Console output:

      ...
      [Pipeline] zip
      Writing zip file of /var/build/workspace/job_branch-123random123/some/dir to /var/build/workspace/job_branch-123random123/my.zip
      Zipped 7 entries.
      Archiving /var/build/workspace/job_branch-123random123/my.zip
      Retrying upload after: java.io.FileNotFoundException: /var/build/workspace/job_branch-123random123/var/build/workspace/job_branch-123random123/my.zip (No such file or directory)
      
      ### Retry repeats many times ###
      
      Retrying upload after: java.io.FileNotFoundException: /var/build/workspace/job_branch-123random123/var/build/workspace/job_branch-123random123/my.zip (No such file or directory)
      Aborted by User Name
      [Pipeline] }
      ...
      

        Attachments

          Activity

          Hide
          reinholdfuereder Reinhold Füreder added a comment - - edited

          I just stumbled over this as well, but in the meantime the problem has changed:

          • The "zipping" still works like before (and as expected), but for the "archiving" while it logs out the correct archive file path, it then:
            1. actually does not archive AND in addition 
            2. also does not fail
          • actually the archiving behaviour must have changed since then!?
          • (and not necessarily to the better, because it used to be more defensive and fail fast before)

          The culprit code seems to be that:

          1. There are slight differences for file path calculation
            1. zipping:
            2. archiving:
          2. But I guess the problem is that "archiveArtifact" step only supports relative path, or only looks inside the current workspace (maybe due to security reasons!? e.g. do not allow to publish files from anywhere else!?)

           

          node {
              writeFile file: 'hello.txt', text: 'Hello Outer World!'
              dir('hello') {
                  dir('hello2') {
                      writeFile file: 'hello.txt', text: 'Hello World!'
                  }
              }
              sh 'rm *.zip'
              zip zipFile: "${WORKSPACE}/absolute-hello-no-archive.zip", dir: 'hello'
              zip zipFile: 'relative-hello-no-archive.zip', dir: 'hello'
              zip zipFile: "${WORKSPACE}/absolute-hello.zip", dir: 'hello', archive: true
              zip zipFile: 'relative-hello.zip', dir: 'hello', archive: true
              sh 'ls -la'
          }
          

          Leads to this build log:

          Running on Jenkins in /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline
          [Pipeline] {
          [Pipeline] writeFile
          [Pipeline] dir
          Running in /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello
          [Pipeline] {
          [Pipeline] dir
          Running in /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello/hello2
          [Pipeline] {
          [Pipeline] writeFile
          [Pipeline] }
          [Pipeline] // dir
          [Pipeline] }
          [Pipeline] // dir
          [Pipeline] sh
          + rm absolute-hello-no-archive.zip absolute-hello.zip relative-hello-no-archive.zip
          [Pipeline] zip
          Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/absolute-hello-no-archive.zip
          Zipped 2 entries.
          [Pipeline] zip
          Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/relative-hello-no-archive.zip
          Zipped 2 entries.
          [Pipeline] zip
          Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/absolute-hello.zip
          Zipped 2 entries.
          Archiving /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/absolute-hello.zip
          [Pipeline] zip
          Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/relative-hello.zip
          Zipped 2 entries.
          Archiving /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/relative-hello.zip
          [Pipeline] sh
          + ls -la
          total 32
          drwxr-xr-x 3 jenkins jenkins 4096 Apr  9 15:02 .
          drwxr-xr-x 8 jenkins jenkins 4096 Apr  9 14:48 ..
          -rw-r--r-- 1 jenkins jenkins  284 Apr  9 15:02 absolute-hello-no-archive.zip
          -rw-r--r-- 1 jenkins jenkins  284 Apr  9 15:02 absolute-hello.zip
          drwxr-xr-x 3 jenkins jenkins 4096 Apr  9 14:42 hello
          -rw-r--r-- 1 jenkins jenkins   18 Apr  9 15:02 hello.txt
          -rw-r--r-- 1 jenkins jenkins  284 Apr  9 15:02 relative-hello-no-archive.zip
          -rw-r--r-- 1 jenkins jenkins  284 Apr  9 15:02 relative-hello.zip
          [Pipeline] }
          [Pipeline] // node
          [Pipeline] End of Pipeline
          Finished: SUCCESS
          

          and only "relative-hello.zip" is archived, but NOT "absolute-hello.zip"!

           

          (Update: Fix groovy syntax for gstring interpolation)

          Show
          reinholdfuereder Reinhold Füreder added a comment - - edited I just stumbled over this as well, but in the meantime the problem has changed: The "zipping" still works like before (and as expected), but for the "archiving" while it logs out the correct archive file path, it then: actually does not archive AND in addition  also does not fail actually the archiving behaviour must have changed since then!? (and not necessarily to the better, because it used to be more defensive and fail fast before) The culprit code seems to be that: There are slight differences for file path calculation zipping: https://github.com/jenkinsci/pipeline-utility-steps-plugin/blob/e76d7fa068b57246e799b9ad181727604a2351ff/src/main/java/org/jenkinsci/plugins/pipeline/utility/steps/zip/ZipStepExecution.java#L83 FilePath destination = ws.child(step.getZipFile()); archiving: https://github.com/jenkinsci/pipeline-utility-steps-plugin/blob/e76d7fa068b57246e799b9ad181727604a2351ff/src/main/java/org/jenkinsci/plugins/pipeline/utility/steps/zip/ZipStepExecution.java#L106 String s = step.getZipFile().replace(...); // (replace args removed) But I guess the problem is that "archiveArtifact" step only supports relative path, or only looks inside the current workspace (maybe due to security reasons!? e.g. do not allow to publish files from anywhere else!?)   node { writeFile file: 'hello.txt' , text: 'Hello Outer World!' dir( 'hello' ) { dir( 'hello2' ) { writeFile file: 'hello.txt' , text: 'Hello World!' } } sh 'rm *.zip' zip zipFile: "${WORKSPACE}/absolute-hello-no-archive.zip" , dir: 'hello' zip zipFile: 'relative-hello-no-archive.zip' , dir: 'hello' zip zipFile: "${WORKSPACE}/absolute-hello.zip" , dir: 'hello' , archive: true zip zipFile: 'relative-hello.zip' , dir: 'hello' , archive: true sh 'ls -la' } Leads to this build log: Running on Jenkins in /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline [Pipeline] { [Pipeline] writeFile [Pipeline] dir Running in /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello [Pipeline] { [Pipeline] dir Running in /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello/hello2 [Pipeline] { [Pipeline] writeFile [Pipeline] } [Pipeline] // dir [Pipeline] } [Pipeline] // dir [Pipeline] sh + rm absolute-hello-no-archive.zip absolute-hello.zip relative-hello-no-archive.zip [Pipeline] zip Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/absolute-hello-no-archive.zip Zipped 2 entries. [Pipeline] zip Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/relative-hello-no-archive.zip Zipped 2 entries. [Pipeline] zip Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/absolute-hello.zip Zipped 2 entries. Archiving /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/absolute-hello.zip [Pipeline] zip Writing zip file of /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/hello to /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/relative-hello.zip Zipped 2 entries. Archiving /var/jenkins_home/workspace/Demo/Demo-Betankung-Pipeline/relative-hello.zip [Pipeline] sh + ls -la total 32 drwxr-xr-x 3 jenkins jenkins 4096 Apr 9 15:02 . drwxr-xr-x 8 jenkins jenkins 4096 Apr 9 14:48 .. -rw-r--r-- 1 jenkins jenkins 284 Apr 9 15:02 absolute-hello-no-archive.zip -rw-r--r-- 1 jenkins jenkins 284 Apr 9 15:02 absolute-hello.zip drwxr-xr-x 3 jenkins jenkins 4096 Apr 9 14:42 hello -rw-r--r-- 1 jenkins jenkins 18 Apr 9 15:02 hello.txt -rw-r--r-- 1 jenkins jenkins 284 Apr 9 15:02 relative-hello-no-archive.zip -rw-r--r-- 1 jenkins jenkins 284 Apr 9 15:02 relative-hello.zip [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS and only "relative-hello.zip" is archived, but NOT "absolute-hello.zip"!   (Update: Fix groovy syntax for gstring interpolation)
          Hide
          reinholdfuereder Reinhold Füreder added a comment -

          So – assuming that the artifact to publish is in the workspace – the workaround seems to only hand over the relative path of the artifact with respect to the workspace, i.e. to strip the  workspace filename from the artifact filename :-|

          Show
          reinholdfuereder Reinhold Füreder added a comment - So – assuming that the artifact to publish is in the workspace – the workaround seems to only hand over the relative path of the artifact with respect to the workspace, i.e. to strip the  workspace filename from the artifact filename :-|
          Hide
          reinholdfuereder Reinhold Füreder added a comment - - edited

          Unfortunately I am not really able to re-produce the problem (on my M$ Windows) in a unit test of the plugin project:

          • A copy of org.jenkinsci.plugins.pipeline.utility.steps.zip.ZipStepTest#simpleArchivedZip
          • ... with change in green like:
            • zip zipFile: "${WORKSPACE}/hello.zip"
          • ...(please mind the double quote chang for gstring interpolation) as it fails with Linux vs. Windows path problem:

          java.nio.file.InvalidPathException: Illegal char <:> at index 76: C:\Users\R3FAB~1.FUE\AppData\Local\Temp\j h5023854380218855900\workspace\p\C:\Users\R3FAB~1.FUE\AppData\Local\Temp\j h5023854380218855900\workspace\p\hello.zip

          However, I guess the reproducer may work under Linux?

          Show
          reinholdfuereder Reinhold Füreder added a comment - - edited Unfortunately I am not really able to re-produce the problem (on my M$ Windows) in a unit test of the plugin project: A copy of org.jenkinsci.plugins.pipeline.utility.steps.zip.ZipStepTest#simpleArchivedZip ... with change in green like: zip zipFile: "${WORKSPACE}/ hello.zip " ...(please mind the double quote chang for gstring interpolation) as it fails with Linux vs. Windows path problem: java.nio.file.InvalidPathException: Illegal char <:> at index 76: C:\Users\R3FAB~1.FUE\AppData\Local\Temp\j h5023854380218855900\workspace\p\C:\Users\R3FAB~1.FUE\AppData\Local\Temp\j h5023854380218855900\workspace\p\hello.zip However, I guess the reproducer may work under Linux?

            People

            Assignee:
            rsandell rsandell
            Reporter:
            mark999 Mark Pictor
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated: