[JENKINS-32832] Some symlinks are missing from the copied artifacts

Type: Bug
Resolution: Fixed
Priority: Major
Component/s: copyartifact-plugin
Labels:
None
Environment:
Copy Artifact Plugin 1.37 on Jenkins 1.646, on Ubuntu 12.04

Similar Issues:
Powered by SuggestiMate

Show

Hi,

I ran into the issue described in this thread:

http://jenkins-ci.361315.n4.nabble.com/Strange-issue-with-copy-artifact-plugin-not-copying-everything-td4756314.html
I can't see any errors in the console output of the build.

Regards,
Robert

quotes from the thread

I created a brand new freestyle job that ALL it does is copy the
artifacts. The first time it is run, only 15 are copied. If I click build
now again, then the missing ones are copied. If I delete the workspace,
the same pattern happens again.

In case anyone has thoughts, here is the parent job:

Centos 6 installed packages:
yum -y install gcc-c++ binutils scons chrpath tar python-devel

Tar from:
http://download-mirror.savannah.gnu.org/releases/gpsd/

Build step:
tar -xzf gpsd-3.10.tar.gz
cd gpsd-3.10
echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct
scons prefix=$WORKSPACE/stage myinstall

Archive artifacts: stage/*/

This is my test job:

Copy Artifact plugin
Project name: libgps
Which build: Latest successful build
Target directory: libgps

Missing files in libgps/stage/lib (Always the same):
libgps.so
libgps.so.21
libgps.so.21.0

As another workaround (besides running the job twice) you can add a second
"Copy Artifacts" build step with the exact same settings and it will work.

Robert Fekete created issue - 2016-02-08 11:40

ikedam made changes - 2016-07-10 02:08

Description

Original: Hi,

I ran into the issue described in this thread:

http://jenkins-ci.361315.n4.nabble.com/Strange-issue-with-copy-artifact-plugin-not-copying-everything-td4756314.html
I can't see any errors in the console output of the build.

Regards,
Robert

New: Hi,

I ran into the issue described in this thread:

http://jenkins-ci.361315.n4.nabble.com/Strange-issue-with-copy-artifact-plugin-not-copying-everything-td4756314.html
I can't see any errors in the console output of the build.

Regards,
Robert

----

quotes from the thread
{quote}
I created a brand new freestyle job that ALL it does is copy the
artifacts. The first time it is run, only 15 are copied. If I click build
now again, then the missing ones are copied. If I delete the workspace,
the same pattern happens again.

In case anyone has thoughts, here is the parent job:

Centos 6 installed packages:
yum -y install gcc-c++ binutils scons chrpath tar python-devel

Tar from:
http://download-mirror.savannah.gnu.org/releases/gpsd/

Build step:
tar -xzf gpsd-3.10.tar.gz
cd gpsd-3.10
echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct
scons prefix=$WORKSPACE/stage myinstall

Archive artifacts: stage/**/*

This is my test job:

Copy Artifact plugin
Project name: libgps
Which build: Latest successful build
Target directory: libgps

Missing files in libgps/stage/lib (Always the same):
libgps.so
libgps.so.21
libgps.so.21.0

As another workaround (besides running the job twice) you can add a second
"Copy Artifacts" build step with the exact same settings and it will work.
{quote}

ikedam added a comment - 2016-07-10 02:18

Reproduced in my environment: ubuntu12.04.5 (x64) + java7u101 + jenkins 2.7.1 + copyartifact 1.38.

setup a new ubuntu server 12.04.5 LTS amd64.
install jenkins as described in Installing+Jenkins+on+Ubuntu

install required packages to build the software

sudo apt-get install scons
sudo apt-get install build-essential
sudo apt-get install chrpath
sudo apt-get install libncurses-dev

Install copyartifact

Create a upstream project:

Execute shell

rm -rf *
wget -q http://download-mirror.savannah.gnu.org/releases/gpsd/gpsd-3.10.tar.gz
tar -xzf gpsd-3.10.tar.gz
cd gpsd-3.10
echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct
scons prefix=$WORKSPACE/stage myinstall

Archive the artifacts
- Files to archive: stage/**/*

Run a build. Following files are created:

ls -laR /var/lib/jenkins/jobs/upstream/builds/1/archive
...
/var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib:
total 484
lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:13 libgpsd.so -> libgpsd.so.22.0.0
lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:13 libgpsd.so.22 -> libgpsd.so.22.0.0
lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:13 libgpsd.so.22.0 -> libgpsd.so.22.0.0
-rw-r--r-- 1 jenkins jenkins 372096 Jul  9 23:13 libgpsd.so.22.0.0
lrwxrwxrwx 1 jenkins jenkins     16 Jul  9 23:13 libgps.so -> libgps.so.21.0.0
lrwxrwxrwx 1 jenkins jenkins     16 Jul  9 23:13 libgps.so.21 -> libgps.so.21.0.0
lrwxrwxrwx 1 jenkins jenkins     16 Jul  9 23:13 libgps.so.21.0 -> libgps.so.21.0.0
-rw-r--r-- 1 jenkins jenkins 122312 Jul  9 23:13 libgps.so.21.0.0
...

Create a downstream project:
- Execute shell
```
rm -rf *
```
- Copy artifacts from another project
  - Project name: upstream
  - Which build: Last successful build
  - Artifacts to copy: **/*

Run a build. Following files are created:

ls -laR /var/lib/jenkins/workspace/downstream
...
./stage/lib:
total 484
lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:15 libgpsd.so -> libgpsd.so.22.0.0
lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:15 libgpsd.so.22 -> libgpsd.so.22.0.0
lrwxrwxrwx 1 jenkins jenkins     17 Jul  9 23:15 libgpsd.so.22.0 -> libgpsd.so.22.0.0
-rw-r--r-- 1 jenkins jenkins 372096 Jul  9 23:13 libgpsd.so.22.0.0
-rw-r--r-- 1 jenkins jenkins 122312 Jul  9 23:13 libgps.so.21.0.0
...

ikedam added a comment - 2016-07-10 02:18 Reproduced in my environment: ubuntu12.04.5 (x64) + java7u101 + jenkins 2.7.1 + copyartifact 1.38. setup a new ubuntu server 12.04.5 LTS amd64. install jenkins as described in Installing+Jenkins+on+Ubuntu install required packages to build the software sudo apt-get install scons sudo apt-get install build-essential sudo apt-get install chrpath sudo apt-get install libncurses-dev Install copyartifact Create a upstream project: Execute shell rm -rf * wget -q http://download-mirror.savannah.gnu.org/releases/gpsd/gpsd-3.10.tar.gz tar -xzf gpsd-3.10.tar.gz cd gpsd-3.10 echo "env.Alias('myinstall', binaryinstall + headerinstall)" >> SConstruct scons prefix=$WORKSPACE/stage myinstall Archive the artifacts Files to archive: stage/**/* Run a build. Following files are created: ls -laR /var/lib/jenkins/jobs/upstream/builds/1/archive ... /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib: total 484 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:13 libgpsd.so -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:13 libgpsd.so.22 -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:13 libgpsd.so.22.0 -> libgpsd.so.22.0.0 -rw-r--r-- 1 jenkins jenkins 372096 Jul 9 23:13 libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 16 Jul 9 23:13 libgps.so -> libgps.so.21.0.0 lrwxrwxrwx 1 jenkins jenkins 16 Jul 9 23:13 libgps.so.21 -> libgps.so.21.0.0 lrwxrwxrwx 1 jenkins jenkins 16 Jul 9 23:13 libgps.so.21.0 -> libgps.so.21.0.0 -rw-r--r-- 1 jenkins jenkins 122312 Jul 9 23:13 libgps.so.21.0.0 ... Create a downstream project: Execute shell rm -rf * Copy artifacts from another project Project name: upstream Which build: Last successful build Artifacts to copy: **/* Run a build. Following files are created: ls -laR /var/lib/jenkins/workspace/downstream ... ./stage/lib: total 484 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:15 libgpsd.so -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:15 libgpsd.so.22 -> libgpsd.so.22.0.0 lrwxrwxrwx 1 jenkins jenkins 17 Jul 9 23:15 libgpsd.so.22.0 -> libgpsd.so.22.0.0 -rw-r--r-- 1 jenkins jenkins 372096 Jul 9 23:13 libgpsd.so.22.0.0 -rw-r--r-- 1 jenkins jenkins 122312 Jul 9 23:13 libgps.so.21.0.0 ...

ikedam made changes - 2016-07-10 02:19

Summary

Original: Some files are missing from the copied artifacts

New: Some symlinks are missing from the copied artifacts

ikedam added a comment - 2016-07-10 02:22

What happens internally (created a special package to output verbose logs):

Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so
ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so failed: 2 No such file or directory
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21
ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21 failed: 2 No such file or directory
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0
ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0 failed: 2 No such file or directory
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0.0
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0
Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0.0

It failed to create some symlinks as the directory (/var/lib/jenkins/workspace/downstream/stage/lib) was not available.
Once a non-symlink file in the directory is copied, the directory is created and starts to succeed to create symlinks.

ikedam added a comment - 2016-07-10 02:22 What happens internally (created a special package to output verbose logs): Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so failed: 2 No such file or directory Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21 ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21 failed: 2 No such file or directory Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0 ln -s libgps.so.21.0.0 /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0 failed: 2 No such file or directory Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgps.so.21.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgps.so.21.0.0 Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22 Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0 Copying /var/lib/jenkins/jobs/upstream/builds/1/archive/stage/lib/libgpsd.so.22.0.0 -> /var/lib/jenkins/workspace/downstream/stage/lib/libgpsd.so.22.0.0 It failed to create some symlinks as the directory (/var/lib/jenkins/workspace/downstream/stage/lib) was not available. Once a non-symlink file in the directory is copied, the directory is created and starts to succeed to create symlinks.

ikedam made changes - 2016-07-10 02:22

Assignee

New: ikedam [ ikedam ]

ikedam made changes - 2016-07-10 03:27

Status

Original: Open [ 1 ]

New: In Progress [ 3 ]

ikedam added a comment - 2016-07-10 03:27

https://github.com/jenkinsci/copyartifact-plugin/pull/86

ikedam added a comment - 2016-07-10 03:27 https://github.com/jenkinsci/copyartifact-plugin/pull/86

Wei Ren added a comment - 2016-07-10 05:45

I am also seeing this issue on my two jenkins sites. One jenkins setup is Jenkins 1.651.3 + Copy Artifact Plugin 1.38, and the other jenkins setup is Jenkins 2.7.1 + Copy Aritifact Plugin 1.38. The OS of both setups is Ubuntu 14.04 LTS.

Below are repro steps:
1. One project P1 archives one file plus one symlink as its build artifacts like:
directory1/file1, directory2/symlink2 where symlink2 is a link to ../directory1/file1.

2. Another project P2 copies P1's artifacts to its workspace at the beginning of its build.

Expected: both file1 and symlink2 are copied to P2's workspace.

Actual: only file1 is copied to P2's workspace.

Please let me know if you need more information.

Wei Ren added a comment - 2016-07-10 05:45 I am also seeing this issue on my two jenkins sites. One jenkins setup is Jenkins 1.651.3 + Copy Artifact Plugin 1.38, and the other jenkins setup is Jenkins 2.7.1 + Copy Aritifact Plugin 1.38. The OS of both setups is Ubuntu 14.04 LTS. Below are repro steps: 1. One project P1 archives one file plus one symlink as its build artifacts like: directory1/file1, directory2/symlink2 where symlink2 is a link to ../directory1/file1. 2. Another project P2 copies P1's artifacts to its workspace at the beginning of its build. Expected: both file1 and symlink2 are copied to P2's workspace. Actual: only file1 is copied to P2's workspace. Please let me know if you need more information.

ikedam added a comment - 2016-07-24 00:55

Fixed with copyartifact-1.38.1.
It will be available in the update center in a day.

frobert renwei
Please try that.

ikedam added a comment - 2016-07-24 00:55 Fixed with copyartifact-1.38.1. It will be available in the update center in a day. frobert renwei Please try that.

ikedam made changes - 2016-07-24 00:55

Assignee	Original: ikedam [ ikedam ]	New: Robert Fekete [ frobert ]
Resolution		New: Fixed [ 1 ]
Status	Original: In Progress [ 3 ]	New: Resolved [ 5 ]

Assignee:: Robert Fekete

Reporter:: Robert Fekete

Votes:: 1 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: 2016-02-08 11:40

Updated:: 2020-05-23 01:19

Resolved:: 2016-07-24 00:55

Details

Description

Attachments

Activity

Collapse comment: ikedam added a comment - 2016-07-10 02:18

Expand comment: ikedam added a comment - 2016-07-10 02:18

Collapse comment: ikedam added a comment - 2016-07-10 02:22

Expand comment: ikedam added a comment - 2016-07-10 02:22

Collapse comment: ikedam added a comment - 2016-07-10 03:27

Expand comment: ikedam added a comment - 2016-07-10 03:27

Collapse comment: Wei Ren added a comment - 2016-07-10 05:45

Expand comment: Wei Ren added a comment - 2016-07-10 05:45

Collapse comment: ikedam added a comment - 2016-07-24 00:55

Expand comment: ikedam added a comment - 2016-07-24 00:55

People

Dates