Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-21958

ArtifactManager introduces regression with publishing symbolic links.

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core

      I have observed a regression issue with the introduction of the new ArtifactManager (https://github.com/jenkinsci/jenkins/commit/e1eea67bb2fa1a356492e91313d84a523f441b34 - https://issues.jenkins-ci.org/browse/JENKINS-17236 ).

      I have pinpointed the issue and saw it was introduced in jenkins 1.532 (as in it works perfectly in 1.531, but fails in 1.532 and above, including the 1.532.2 LTS version).

      My particular use case is that the result of our build creates a symlink which points to the final artifact (the final artifact as stored on disk has a timestamp/build number in its name). However with the introduction of the new ArtifactManager the artifacts which are identified using symlinks no longer get published.

      Steps to reproduce:

      • Create a new job
      • Add a shell build step
        rm -rf folder
        mkdir folder 
        echo "bla" > folder/artifact
        ln -sf ./artifact folder/link
        
      • Add a 'Archive the Artifacts' post-build action and in the Files to archive text box enter
        folder/link
        

      Expected behavior (as implemented/verified) on jenkins 1.524 - 1.531:

      • The build succeeds
      • The 'Last Successful Artifacts' mentions:
        link 4B
        

      Actual behavior (as implemented/verified) on jenkins 1.532 - 1.552, including LTS:

      • The build succeeds
      • The 'Last Successful Artifacts' mentions nothing.

      (for reference the build output:

      Started by user anonymous
      Building on master in workspace /home/jenkins/jenkins/jobs/brossel_edb/workspace
      [workspace] $ /bin/sh -xe /tmp/hudson1352860404870483249.sh
      + rm -rf folder
      + mkdir folder
      + echo bla
      + ln -sf ./artifact folder/link
      Archiving artifacts
      Finished: SUCCESS
      

      )

      An additional sidenote is that the ArtifactDeployer plugin (version 0.28) is doing its job properly, another indication that the way symlinks are resolved/dealt with has likely changed due the introduction of the new ArtifactManager.

          [JENKINS-21958] ArtifactManager introduces regression with publishing symbolic links.

          Jesse Glick added a comment -

          The problem arises in the particular case that the directory containing the symlink was not otherwise archived. (Which would be odd in the case you cite, since there is not much purpose in archiving link without also archiving artifact, since the link would be dead, but there could be other cases where a directory containing just symlinks is being archived for some reason along with other directories with the target files.)

          Jesse Glick added a comment - The problem arises in the particular case that the directory containing the symlink was not otherwise archived. (Which would be odd in the case you cite, since there is not much purpose in archiving link without also archiving artifact , since the link would be dead, but there could be other cases where a directory containing just symlinks is being archived for some reason along with other directories with the target files.)

          Thanks for the swift investigation. As far as I can tell from my tests, the pre-1.532 did actually a deep copy of the contents of the symlink, hence instead of archiving the symlink it archived the contents of the link under the name of the link.

          (In my particular case I would have a folder having
          build-1.bin
          build-2.bin
          build-3.bin
          build-4.bin
          latest.bin (which points to build-4.bin) )

          When I would point jenkins to 'latest.bin' (since the counter in my scenario is not know/set by jenkins) it would actually create a file called 'latest.bin' with the contents of build-4.bin, aka a deep-copy.

          (I'm not saying that this behavior is 100% sane/correct/understandable, since obviously this may result in unwanted duplication of data when not done carefully, but in my case it did what it was supposed to do, and for that account the artefactdeployer plugin was doing exactly the same thing).

          Elie De Brauwer added a comment - Thanks for the swift investigation. As far as I can tell from my tests, the pre-1.532 did actually a deep copy of the contents of the symlink, hence instead of archiving the symlink it archived the contents of the link under the name of the link. (In my particular case I would have a folder having build-1.bin build-2.bin build-3.bin build-4.bin latest.bin (which points to build-4.bin) ) When I would point jenkins to 'latest.bin' (since the counter in my scenario is not know/set by jenkins) it would actually create a file called 'latest.bin' with the contents of build-4.bin, aka a deep-copy. (I'm not saying that this behavior is 100% sane/correct/understandable, since obviously this may result in unwanted duplication of data when not done carefully, but in my case it did what it was supposed to do, and for that account the artefactdeployer plugin was doing exactly the same thing).

          Jesse Glick added a comment -

          pre-1.532 did actually a deep copy of the contents of the symlink

          If true, I would consider that a bug in the older version. The principle of least surprise means that if your workspace has a mixture of regular files and symlinks, and you ask to archive some or all of them, the resulting artifact directory contains the exact same things.

          If you wished to archive a particular regular file that might change from build to build, then you must have a build step which copies it to some fixed location.

          Jesse Glick added a comment - pre-1.532 did actually a deep copy of the contents of the symlink If true, I would consider that a bug in the older version. The principle of least surprise means that if your workspace has a mixture of regular files and symlinks, and you ask to archive some or all of them, the resulting artifact directory contains the exact same things. If you wished to archive a particular regular file that might change from build to build, then you must have a build step which copies it to some fixed location.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/FilePath.java
          test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
          http://jenkins-ci.org/commit/jenkins/7c568178f3810ab590d33ba23f1cf7fea418edac
          Log:
          [FIXED JENKINS-21958] FilePath.copyRecursiveTo given a scanner including a symlink but not any regular file in the same directory would neglect to create the parent directory on the destination and thus not copy the link.
          (Util.createSymlink is partly to blame for not throwing a descriptive IOException when it fails to create a link,
          but this is its historical behavior needed for compatibility.
          copyRecursiveTo also continues to suppress error text from this method because there is nowhere for it to go.
          Perhaps need a new variant of createSymlink that is guaranteed to either have created the link or throw an exception.)

          Compare: https://github.com/jenkinsci/jenkins/compare/ddbba3405434...7c568178f381

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/FilePath.java test/src/test/java/hudson/tasks/ArtifactArchiverTest.java http://jenkins-ci.org/commit/jenkins/7c568178f3810ab590d33ba23f1cf7fea418edac Log: [FIXED JENKINS-21958] FilePath.copyRecursiveTo given a scanner including a symlink but not any regular file in the same directory would neglect to create the parent directory on the destination and thus not copy the link. (Util.createSymlink is partly to blame for not throwing a descriptive IOException when it fails to create a link, but this is its historical behavior needed for compatibility. copyRecursiveTo also continues to suppress error text from this method because there is nowhere for it to go. Perhaps need a new variant of createSymlink that is guaranteed to either have created the link or throw an exception.) Compare: https://github.com/jenkinsci/jenkins/compare/ddbba3405434...7c568178f381

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3197
          [FIXED JENKINS-21958] FilePath.copyRecursiveTo given a scanner including a symlink but not any regular file in the same directory would neglect to create the parent directory on the destination and thus not copy the link. (Revision 7c568178f3810ab590d33ba23f1cf7fea418edac)

          Result = UNSTABLE
          Jesse Glick : 7c568178f3810ab590d33ba23f1cf7fea418edac
          Files :

          • core/src/main/java/hudson/FilePath.java
          • test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #3197 [FIXED JENKINS-21958] FilePath.copyRecursiveTo given a scanner including a symlink but not any regular file in the same directory would neglect to create the parent directory on the destination and thus not copy the link. (Revision 7c568178f3810ab590d33ba23f1cf7fea418edac) Result = UNSTABLE Jesse Glick : 7c568178f3810ab590d33ba23f1cf7fea418edac Files : core/src/main/java/hudson/FilePath.java test/src/test/java/hudson/tasks/ArtifactArchiverTest.java changelog.html

          Thanks !

          Elie De Brauwer added a comment - Thanks !

            jglick Jesse Glick
            eliedebrauwer Elie De Brauwer
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: