Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56249

userContent *zip* (all files in zip) stopped working at 2.164

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • Windows 2008 R2 Enterprise, Jenkins 2.164+

      When using the (all files in zip) functionality starting at 2.164, either from the UI or URL (https:/myjenkins/userContent/*zip*/image.zip), the resulting image.zip has no content. 

      This has silently broken our builds, which utilize this functionality to retrieve certain content. Downgrading jenkins.war to 2.163 restores the functionality.

          [JENKINS-56249] userContent *zip* (all files in zip) stopped working at 2.164

          Matt Sicker added a comment -

          Me or one of my teammates will be following up on this ticket to fix the problem. I believe I've been able to isolate the issue down to a difference in behavior when using the escape hatch where you can view files that are links from outside userContent or are themselves descendants of a link, but the same files are skipped in the zip file output regardless of the escape hatch.

          Matt Sicker added a comment - Me or one of my teammates will be following up on this ticket to fix the problem. I believe I've been able to isolate the issue down to a difference in behavior when using the escape hatch where you can view files that are links from outside userContent or are themselves descendants of a link, but the same files are skipped in the zip file output regardless of the escape hatch.

          Same issue here, we use symlinks a lot and are not able to upgrade at the moment to the latest LTS because of this issue. Is there a temporary workaround that can be made on a local Jenkins install? 

          Arjo Poldervaart added a comment - Same issue here, we use symlinks a lot and are not able to upgrade at the moment to the latest LTS because of this issue. Is there a temporary workaround that can be made on a local Jenkins install? 

          Jeff Thompson added a comment -

          arjo_poldervaart, could you clarify some of your use cases for symlinks? How do you use them? Where do they link from and to? Do they link from inside a workspace to somewhere else on the system?

          Anyone else, please respond also, about your need for symlink support.

          I'm especially interested in how your symlink use works with zip files, as featured in the OP. Zip files generated by Jenkins do not contain symlinks but may resolve the symlink to the target and include all of that.

          One scenario, described in JENKINS-56114, involves a master or agent which resides in a path containing a symlink at the root or somewhere else higher in the path.

          Jeff Thompson added a comment - arjo_poldervaart , could you clarify some of your use cases for symlinks? How do you use them? Where do they link from and to? Do they link from inside a workspace to somewhere else on the system? Anyone else, please respond also, about your need for symlink support. I'm especially interested in how your symlink use works with zip files, as featured in the OP. Zip files generated by Jenkins do not contain symlinks but may resolve the symlink to the target and include all of that. One scenario, described in JENKINS-56114 , involves a master or agent which resides in a path containing a symlink at the root or somewhere else higher in the path.

          Jeff Thompson added a comment -

          And which of those use cases involve userContent, as described in the OP?

          Jeff Thompson added a comment - And which of those use cases involve userContent, as described in the OP?

          Glenn Herbert added a comment -

          Our use of (Windows) symlinks (directory junctions) is purely for directories. We store build artifacts on a corporate (remote) NAS drive (multi-TB), but use the jenkins userContent and its built-in file server as a way to 1) download artifacts via jenkins URL (ie curl http:/myJenkins/userContent/symlink/path/to/artifact), or 2) navigate to specific artifacts.  We don't ever request a zip on the actual symlinked folder but some content that resides within that symlinked folder tree.

           

          I would not expect a zip file to contain a symlink but rather its resolved target.

           

          Glenn Herbert added a comment - Our use of (Windows) symlinks (directory junctions) is purely for directories. We store build artifacts on a corporate (remote) NAS drive (multi-TB), but use the jenkins userContent and its built-in file server as a way to 1) download artifacts via jenkins URL (ie curl http:/myJenkins/userContent/symlink/path/to/artifact), or 2) navigate to specific artifacts.  We don't ever request a zip on the actual symlinked folder but some content that resides within that symlinked folder tree.   I would not expect a zip file to contain a symlink but rather its resolved target.  

          Arjo Poldervaart added a comment - - edited

          Our use is similar to that of Glenn. To provide some context:

          If we want to release an artifact to production, we tag it in our SCM and let Jenkins build it, Jenkins then copies the artifact to a different location so we can download it at a later time (even if the workspace is cleared). The copy is done via a symlink created in the workspace by our build script (soft link on Linux) to a folder outside the workspace on disk.

          When we want to deploy the artifact we can simply download it via een link like: /view/assets/job/assets-BootstrapPortlet/ws/RELEASES/1.0.2/BootstrapPortletEAR.ear

          The _RELEASES_ folder __ is a symlink. So we basically use Jenkins as a file server just like Glenn.

           

          We do not use symlinks in the userContent folder, but we do use it in the job workspace.

          Arjo Poldervaart added a comment - - edited Our use is similar to that of Glenn. To provide some context: If we want to release an artifact to production, we tag it in our SCM and let Jenkins build it, Jenkins then copies the artifact to a different location so we can download it at a later time (even if the workspace is cleared). The copy is done via a symlink created in the workspace by our build script (soft link on Linux) to a folder outside the workspace on disk. When we want to deploy the artifact we can simply download it via een link like: /view/assets/job/assets-BootstrapPortlet/ws/ RELEASES /1.0.2/BootstrapPortletEAR.ear The _ RELEASES _ folder __ is a symlink. So we basically use Jenkins as a file server just like Glenn.   We do not use symlinks in the userContent folder, but we do use it in the job workspace.

          Jeff Thompson added a comment -

          kansasmann and arjo_poldervaart, thanks for your responses on describing your use models. That helps clarify some things.

          Have you read the security advisory from last year at https://jenkins.io/security/advisory/2018-12-05/ ? Have you tried disabling that fix via the system property described there? It sounds like your use cases are some of the rare situations where that might make sense.

          Jeff Thompson added a comment - kansasmann and arjo_poldervaart , thanks for your responses on describing your use models. That helps clarify some things. Have you read the security advisory from last year at  https://jenkins.io/security/advisory/2018-12-05/  ? Have you tried disabling that fix via the system property described there? It sounds like your use cases are some of the rare situations where that might make sense.

          Hi Jeff,

          We have been running with "-Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" for quite a while already. The setting does not resolve the current issue.

          Arjo Poldervaart added a comment - Hi Jeff, We have been running with "-Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" for quite a while already. The setting does not resolve the current issue.

          Glenn Herbert added a comment -

          Same here, been running with SymlinkEscape=true for quite a while.

          Glenn Herbert added a comment - Same here, been running with SymlinkEscape=true for quite a while.

          Jeff Thompson added a comment -

          I'm able to reproduce Matt's simple test case on Unix and Windows. Hopefully it's a reasonably good representation of the other problems people are experiencing. Looks like the `allowSymlinkEscape` property isn't working the way it was intended. I'll see if I can figure something out to make it work better. It may take a while to get that change put together.

          Jeff Thompson added a comment - I'm able to reproduce Matt's simple test case on Unix and Windows. Hopefully it's a reasonably good representation of the other problems people are experiencing. Looks like the `allowSymlinkEscape` property isn't working the way it was intended. I'll see if I can figure something out to make it work better. It may take a while to get that change put together.

            Unassigned Unassigned
            kansasmann Glenn Herbert
            Votes:
            2 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: