Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32832

Some symlinks are missing from the copied artifacts

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • copyartifact-plugin
    • None
    • Copy Artifact Plugin 1.37 on Jenkins 1.646, on Ubuntu 12.04

      Hi,

      I ran into the issue described in this thread:

      http://jenkins-ci.361315.n4.nabble.com/Strange-issue-with-copy-artifact-plugin-not-copying-everything-td4756314.html
      I can't see any errors in the console output of the build.

      Regards,
      Robert


      quotes from the thread

      I created a brand new freestyle job that ALL it does is copy the
      artifacts. The first time it is run, only 15 are copied. If I click build
      now again, then the missing ones are copied. If I delete the workspace,
      the same pattern happens again.

      In case anyone has thoughts, here is the parent job:

      Centos 6 installed packages:
      yum -y install gcc-c++ binutils scons chrpath tar python-devel

      Tar from:
      http://download-mirror.savannah.gnu.org/releases/gpsd/

      Build step:
      tar -xzf gpsd-3.10.tar.gz
      cd gpsd-3.10
      echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct
      scons prefix=$WORKSPACE/stage myinstall

      Archive artifacts: stage/*/

      This is my test job:

      Copy Artifact plugin
      Project name: libgps
      Which build: Latest successful build
      Target directory: libgps

      Missing files in libgps/stage/lib (Always the same):
      libgps.so
      libgps.so.21
      libgps.so.21.0

      As another workaround (besides running the job twice) you can add a second
      "Copy Artifacts" build step with the exact same settings and it will work.

          [JENKINS-32832] Some symlinks are missing from the copied artifacts

          ikedam added a comment -

          Reproduced in my environment: ubuntu12.04.5 (x64) + java7u101 + jenkins 2.7.1 + copyartifact 1.38.

          1. setup a new ubuntu server 12.04.5 LTS amd64.
          2. install jenkins as described in Installing+Jenkins+on+Ubuntu
          3. install required packages to build the software
            sudo apt-get install scons
            sudo apt-get install build-essential
            sudo apt-get install chrpath
            sudo apt-get install libncurses-dev
            
          4. Install copyartifact
          5. Create a upstream project:
            • Execute shell
              rm -rf *
              wget -q http://download-mirror.savannah.gnu.org/releases/gpsd/gpsd-3.10.tar.gz
              tar -xzf gpsd-3.10.tar.gz
              cd gpsd-3.10
              echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct
              scons prefix=$WORKSPACE/stage myinstall 
              
            • Archive the artifacts
              • Files to archive: stage/**/*
          6. Run a build. Following files are created:
            ls -laR /var/lib/jenkins/jobs/upstream/builds/1/archive
            ...
            /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib:
            total 484
            lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:13 libgpsd.so -> libgpsd.so.22.0.0
            lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:13 libgpsd.so.22 -> libgpsd.so.22.0.0
            lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:13 libgpsd.so.22.0 -> libgpsd.so.22.0.0
            -rw-r--r-- 1 jenkins jenkins 372096 Jul  9 23:13 libgpsd.so.22.0.0
            lrwxrwxrwx 1 jenkins jenkins     16 Jul  9 23:13 libgps.so -> libgps.so.21.0.0
            lrwxrwxrwx 1 jenkins jenkins     16 Jul  9 23:13 libgps.so.21 -> libgps.so.21.0.0
            lrwxrwxrwx 1 jenkins jenkins     16 Jul  9 23:13 libgps.so.21.0 -> libgps.so.21.0.0
            -rw-r--r-- 1 jenkins jenkins 122312 Jul  9 23:13 libgps.so.21.0.0
            ...
            
          7. Create a downstream project:
            • Execute shell
              rm -rf *
              
            • Copy artifacts from another project
              • Project name: upstream
              • Which build: Last successful build
              • Artifacts to copy: **/*
          8. Run a build. Following files are created:
            ls -laR /var/lib/jenkins/workspace/downstream
            ...
            ./stage/lib:
            total 484
            lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:15 libgpsd.so -> libgpsd.so.22.0.0
            lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:15 libgpsd.so.22 -> libgpsd.so.22.0.0
            lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:15 libgpsd.so.22.0 -> libgpsd.so.22.0.0
            -rw-r--r-- 1 jenkins jenkins 372096 Jul  9 23:13 libgpsd.so.22.0.0
            -rw-r--r-- 1 jenkins jenkins 122312 Jul  9 23:13 libgps.so.21.0.0
            ...
            

          ikedam added a comment - Reproduced in my environment: ubuntu12.04.5 (x64) + java7u101 + jenkins 2.7.1 + copyartifact 1.38. setup a new ubuntu server 12.04.5 LTS amd64. install jenkins as described in Installing+Jenkins+on+Ubuntu install required packages to build the software sudo apt-get install scons sudo apt-get install build-essential sudo apt-get install chrpath sudo apt-get install libncurses-dev Install copyartifact Create a upstream project: Execute shell rm -rf * wget -q http://download-mirror.savannah.gnu.org/releases/gpsd/gpsd-3.10.tar.gz tar -xzf gpsd-3.10.tar.gz cd gpsd-3.10 echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct scons prefix=$WORKSPACE/stage myinstall Archive the artifacts Files to archive: stage/**/* Run a build. Following files are created: ls -laR /var/lib/jenkins/jobs/upstream/builds/1/archive ... /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib: total 484 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:13 libgpsd.so -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:13 libgpsd.so.22 -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:13 libgpsd.so.22.0 -> libgpsd.so.22.0.0 -rw-r--r-- 1 jenkins jenkins 372096 Jul 9 23:13 libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 16 Jul 9 23:13 libgps.so -> libgps.so.21.0.0 lrwxrwxrwx 1 jenkins jenkins 16 Jul 9 23:13 libgps.so.21 -> libgps.so.21.0.0 lrwxrwxrwx 1 jenkins jenkins 16 Jul 9 23:13 libgps.so.21.0 -> libgps.so.21.0.0 -rw-r--r-- 1 jenkins jenkins 122312 Jul 9 23:13 libgps.so.21.0.0 ... Create a downstream project: Execute shell rm -rf * Copy artifacts from another project Project name: upstream Which build: Last successful build Artifacts to copy: **/* Run a build. Following files are created: ls -laR /var/lib/jenkins/workspace/downstream ... ./stage/lib: total 484 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:15 libgpsd.so -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:15 libgpsd.so.22 -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:15 libgpsd.so.22.0 -> libgpsd.so.22.0.0 -rw-r--r-- 1 jenkins jenkins 372096 Jul 9 23:13 libgpsd.so.22.0.0 -rw-r--r-- 1 jenkins jenkins 122312 Jul 9 23:13 libgps.so.21.0.0 ...

          ikedam added a comment -

          What happens internally (created a special package to output verbose logs):

          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so
          ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so failed: 2 No such file or directory
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21
          ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21 failed: 2 No such file or directory
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0
          ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0 failed: 2 No such file or directory
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0.0
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0
          Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0.0
          

          It failed to create some symlinks as the directory (/var/lib/jenkins/workspace/downstream/stage/lib) was not available.
          Once a non-symlink file in the directory is copied, the directory is created and starts to succeed to create symlinks.

          ikedam added a comment - What happens internally (created a special package to output verbose logs): Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so failed: 2 No such file or directory Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21 ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21 failed: 2 No such file or directory Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0 ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0 failed: 2 No such file or directory Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0.0 Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22 Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0 Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0.0 It failed to create some symlinks as the directory (/var/lib/jenkins/workspace/downstream/stage/lib) was not available. Once a non-symlink file in the directory is copied, the directory is created and starts to succeed to create symlinks.

          ikedam added a comment -

          ikedam added a comment - https://github.com/jenkinsci/copyartifact-plugin/pull/86

          Wei Ren added a comment -

          I am also seeing this issue on my two jenkins sites. One jenkins setup is Jenkins 1.651.3 + Copy Artifact Plugin 1.38, and the other jenkins setup is Jenkins 2.7.1 + Copy Aritifact Plugin 1.38. The OS of both setups is Ubuntu 14.04 LTS.

          Below are repro steps:
          1. One project P1 archives one file plus one symlink as its build artifacts like:
          directory1/file1, directory2/symlink2 where symlink2 is a link to ../directory1/file1.

          2. Another project P2 copies P1's artifacts to its workspace at the beginning of its build.

          Expected: both file1 and symlink2 are copied to P2's workspace.

          Actual: only file1 is copied to P2's workspace.

          Please let me know if you need more information.

          Wei Ren added a comment - I am also seeing this issue on my two jenkins sites. One jenkins setup is Jenkins 1.651.3 + Copy Artifact Plugin 1.38, and the other jenkins setup is Jenkins 2.7.1 + Copy Aritifact Plugin 1.38. The OS of both setups is Ubuntu 14.04 LTS. Below are repro steps: 1. One project P1 archives one file plus one symlink as its build artifacts like: directory1/file1, directory2/symlink2 where symlink2 is a link to ../directory1/file1. 2. Another project P2 copies P1's artifacts to its workspace at the beginning of its build. Expected: both file1 and symlink2 are copied to P2's workspace. Actual: only file1 is copied to P2's workspace. Please let me know if you need more information.

          ikedam added a comment -

          Fixed with copyartifact-1.38.1.
          It will be available in the update center in a day.

          frobert renwei
          Please try that.

          ikedam added a comment - Fixed with copyartifact-1.38.1. It will be available in the update center in a day. frobert renwei Please try that.

          Wei Ren added a comment -

          Sure. Thank you for fixing this. Will give it a try.

          Wei Ren added a comment - Sure. Thank you for fixing this. Will give it a try.

          Wei Ren added a comment -

          Updated the Copy Artifacts plugin to 1.38.1 and verified that this bug was gone.

          Wei Ren added a comment - Updated the Copy Artifacts plugin to 1.38.1 and verified that this bug was gone.

            frobert Robert Fekete
            frobert Robert Fekete
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: