Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64818

Job artifacts zip archive format change breaks external tool

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • core
    • None
    • RedHat Enterprise Linux 7
      Jenkins 2.263.2
      Jenkins 2.263.3

    Description

      Since Jenkins 2.263.2 the format of the artifact Zip archive changed in a way that breaks the Azure DevOps plugin we are using to download Jenkins job artifacts after a job has finished. The archives downloaded from Jenkins 2.263.1 and 2.263.2+ contain exactly the same files (as expected), but differ when using binary comparison. This was tested by downloading artifacts from the same build in version 2.263.1 and 2.263.2/2.263.3. I attached one archive downloaded from 2.263.1 and one archive downloaded from 2.263.3. 

      Since there is no easy workaround we had to roll back to 2.263.1.

      Attachments

        Issue Links

          Activity

            markewaite Mark Waite added a comment -

            It seems that you're relying on a binary comparison of the zip file itself. Isn't that much more brittle than is required?

            The purpose of the zip file is to provide all the files and directories from the source location in the workspace. Relying on the precise ordering of files in the zip archive or precise compression levels or other internal details of the zip file format seems more precise than the zip archive of files in the workspace was ever intended to be. Since the changes in Jenkins 2.263.2 resolved one or more security issues with that change, I doubt that the maintainers will be willing to work to provide that level of compatibility.

            markewaite Mark Waite added a comment - It seems that you're relying on a binary comparison of the zip file itself. Isn't that much more brittle than is required? The purpose of the zip file is to provide all the files and directories from the source location in the workspace. Relying on the precise ordering of files in the zip archive or precise compression levels or other internal details of the zip file format seems more precise than the zip archive of files in the workspace was ever intended to be. Since the changes in Jenkins 2.263.2 resolved one or more security issues with that change, I doubt that the maintainers will be willing to work to provide that level of compatibility.
            bb Benjamin Braner added a comment - - edited

            It surely is way too brittle, but the plugin we are using was not written by me. I also filed a bugreport for that plugin in the Azure DevOps issue tracker already. I just thought it would be a good idea to at least make that hidden change public here.

            I don´t even know if that specific change is breaking the plugin in question or if there is more to it. But it is the only behavioural change between Jenkins 2.263.1 and 2.263.3 I could find.

            bb Benjamin Braner added a comment - - edited It surely is way too brittle, but the plugin we are using was not written by me. I also filed a bugreport for that plugin in the Azure DevOps issue tracker already. I just thought it would be a good idea to at least make that hidden change public here. I don´t even know if that specific change is breaking the plugin in question or if there is more to it. But it is the only behavioural change between Jenkins 2.263.1 and 2.263.3 I could find.
            ianw Ian Williams added a comment - - edited I wonder if this isrelated - Backup plungin in jenkins is not able to restore the backupzip in linux
            john_fo John Forrest added a comment -

            This was reported to Azure Devops and analysed their side. See https://github.com/microsoft/azure-pipelines-tasks/issues/14320. Their analysis is that the problem is in the meta data of the zip file generated by Jenkins. I don't know enough to comment but seems we are currently caught in the middle on this. Only known workaround to date is to stick to old Jenkins builds.

            markewaite do you happen to know who to poke on this?

            john_fo John Forrest added a comment - This was reported to Azure Devops and analysed their side. See https://github.com/microsoft/azure-pipelines-tasks/issues/14320.  Their analysis is that the problem is in the meta data of the zip file generated by Jenkins. I don't know enough to comment but seems we are currently caught in the middle on this. Only known workaround to date is to stick to old Jenkins builds. markewaite  do you happen to know who to poke on this?
            susilkumarj Susil added a comment -

            I got here from https://github.com/microsoft/azure-pipelines-tasks/issues/14320 We are using Jenkins docker image 2.277.4. Builds are completed successfully in Jenkins server and errors during artifacts download in Azure Devops.

             
            /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/artifact-engine/Engine/artifactEngine.js:157
            throw err;
            ^
            ExtractionFailed Error: Unsupported file type "undefined"
            at DecompressZip.extractFile (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:309:11)
            at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:242:28
            at Array.forEach (native)
            at DecompressZip.extractFiles (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:241:11)
            at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:121:21
            at _fulfilled (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:854:54)
            at self.promiseDispatch.done (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:883:30)
            at Promise.promise.promiseDispatch (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:816:13)
            at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:624:44
            at runSingle (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:137:13)
            ##[warning]Can\'t find loc string for key: ExtractionFailed
             

            susilkumarj Susil added a comment - I got here from https://github.com/microsoft/azure-pipelines-tasks/issues/14320  We are using Jenkins docker image 2.277.4. Builds are completed successfully in Jenkins server and errors during artifacts download in Azure Devops.   /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/artifact-engine/Engine/artifactEngine.js:157 throw err; ^ ExtractionFailed Error: Unsupported file type "undefined" at DecompressZip.extractFile (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:309:11) at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:242:28 at Array.forEach (native) at DecompressZip.extractFiles (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:241:11) at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:121:21 at _fulfilled (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:854:54) at self.promiseDispatch.done (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:883:30) at Promise.promise.promiseDispatch (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:816:13) at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:624:44 at runSingle (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:137:13) ## [warning] Can\'t find loc string for key: ExtractionFailed  
            mstfn Stefan added a comment - - edited

            Something appears to have changed with the meta-data in Jenkins generated zip files between these versions. Both the versioning and how external attributes are written.

            This caused an issue with the Azure Devops task dependency decompress-zip 0.3.0 but it appears it is fixed in decompress-zip 0.3.3.

            I confirmed 0.3.0 fails with the given error message when attempting to extract the new  zip files whereas 0.3.3 successfully extracts them. Both with the attached files, and with files from Jenkins 2.289.1

            https://github.com/bower/decompress-zip/issues/71

            It appears decompress-zip deems the new zip files to be from UNIX instead of DOS, assuming the "External attributes" regarding file-type to be in a particular format, and the way it expects this format to describe file-type doesn't match what's in the Jenkins generated zip file, which causes it to fail.

            I have let the Azure Devops team know, and hopefully this issue should be closed by them updating their dependencies in the near future.

             

            mstfn Stefan added a comment - - edited Something appears to have changed with the meta-data in Jenkins generated zip files between these versions. Both the versioning and how external attributes are written. This caused an issue with the Azure Devops task dependency decompress-zip 0.3.0 but it appears it is fixed in decompress-zip 0.3.3 . I confirmed 0.3.0 fails with the given error message when attempting to extract the new  zip files whereas  0.3.3 successfully extracts them. Both with the attached files, and with files from Jenkins 2.289.1 https://github.com/bower/decompress-zip/issues/71 It appears decompress-zip deems the new zip files to be from UNIX instead of DOS, assuming the "External attributes" regarding file-type to be in a particular format, and the way it expects this format to describe file-type doesn't match what's in the Jenkins generated zip file, which causes it to fail. I have let the Azure Devops team know, and hopefully this issue should be closed by them updating their dependencies in the near future.  

            People

              Unassigned Unassigned
              bb Benjamin Braner
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: