-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Powered by SuggestiMate
Starting yesterday, the EnvInject plugin seems to suddenly start using a cached value for ${WORKSPACE}. We have the following value in "Properties Content" under "Inject environment variables to the build process":
PATH=${WORKSPACE}/.env/bin:$PATH
When working properly, in the console output this appears as:
[EnvInject] - Injecting as environment variables the properties content
PATH=${WORKSPACE}/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/bin
Over the past couple of days this has suddenly started appearing as:
[EnvInject] - Injecting as environment variables the properties content
PATH=/Users/Shared/Jenkins/Home/jobs/bidpom.stage/workspace/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/bin
Which is using a workspace path from another job. This causes this job, and all other jobs, to fail. If I restart Jenkins the problem goes away, for awhile, but at some point that exact same path starts getting used by the EnvInject plugin again. This cycle has happened 4 times over the past couple of days, where jobs will run fine for awhile and then suddenly start failing because the path injected changes from:
${WORKSPACE}/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/bin
to:
/Users/Shared/Jenkins/Home/jobs/bidpom.stage/workspace/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/bin
When this first happened we downgraded the plugin from 1.78 to 1.77 but this hasn't addressed the problem.
Any help anyone can provide would be greatly appreciated.
[JENKINS-16233] EnvInject plugin using a cached value for ${WORKSPACE}
JENKINS-15658 was introduced in version 1.73. Therefore, if it is the cause, you have to downgrade to 1.72.
For this issue, it seems not to be a workspace from another job but a shared workspace.
Does this value is specified in another place in the job or in the global configuration?
The value is not specified anywhere else in the job nor in the global configuration. It seems to be caching the workspace value that was used for a previous job.
This is happening right now. We need to restart Jenkins to fix this, but it will no doubt happen again. I've done a little investigation and found that echoing ${WORKSPACE} in a build shell shows the correct value, but EnvInject has injected the wrong value:
[CHECKED] Inject environment variables to the build process
Properties Content:
TEST=${WORKSPACE}
PATH=${WORKSPACE}/.env/bin:$PATH
Execute shell:
printenv
echo ${WORKSPACE}
echo ${PATH}
Console output:
...
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
TEST=/Users/Shared/Jenkins/Home/jobs/bidpom.stage.saucelabs/workspace
PATH=/Users/Shared/Jenkins/Home/jobs/bidpom.stage.saucelabs/workspace/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
[EnvInject] - Variables injected successfully.
...
+ printenv
...
TEST=/Users/Shared/Jenkins/Home/jobs/bidpom.stage.saucelabs/workspace
WORKSPACE=/Users/Shared/Jenkins/Home/jobs/bouncer.prod/workspace
PATH=/Users/Shared/Jenkins/Home/jobs/bidpom.stage.saucelabs/workspace/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
...
+ echo /Users/Shared/Jenkins/Home/jobs/bouncer.prod/workspace
/Users/Shared/Jenkins/Home/jobs/bouncer.prod/workspace
+ echo /Users/Shared/Jenkins/Home/jobs/bidpom.stage.saucelabs/workspace/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
/Users/Shared/Jenkins/Home/jobs/bidpom.stage.saucelabs/workspace/.env/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
So you can see that WORKSPACE is accurate according to Jenkins environment variables, but EnvInject is expanding it to an incorrect value.
Please let me know if there's any additional information that would be useful for the next time this fails.
Does '/Users/Shared/Jenkins/Home/jobs/bouncer.prod/workspace' match a specific slave where the build run?
Code changed in jenkins
User: Gregory Boissinot
Path:
src/main/java/org/jenkinsci/plugins/envinject/EnvInjectBuildWrapper.java
src/main/java/org/jenkinsci/plugins/envinject/EnvInjectBuilder.java
src/main/java/org/jenkinsci/plugins/envinject/EnvInjectListener.java
src/test/java/org/jenkinsci/plugins/envinject/GlobalPropertiesTest.java
http://jenkins-ci.org/commit/envinject-plugin/23a7086f231e0647bfc24daceacaa1f078f5aa7f
Log:
Fix JENKINS-16233
EnvInject 1.80 should fix this issue.
Please let me know if it works in your environment.
Thanks
Were you able to replicate the issue? We have upgraded to 1.80 and still see the issue. I've taken a look at the changes and not sure that 'we set a global workspace at job level' is accurate for what we are doing. We are setting an environment variable to the value including ${WORKSPACE} and instead of getting the job workspace it is the value of a previous job's workspace...
I'm sorry I though the previous fix could fix the issue.
I'm not reproducing the issue. Which type is your job (freestyle, maven, matrix, ...)?
Please feel free to isolate the problem and attach a zip with your Jenkins env?
Without isolating the problem, I don't know how to fix it.
I see the same problem, see https://issues.jenkins-ci.org/browse/JENKINS-16448.
No, but when it does, it's always the same wrong value, i.e. the value of a since long deleted project. In my case the value is always "main-multiconf" which was the value of and old matrix job that was deleted several months ago. It may actually have been the first matrix job we ever created.
I meant that the value is always "/var/lib/jenkins/jobs/main-multiconf/workspace".
We have rolled back to version 1.61 and so far have not seen this issue since.
Is it possible for you to reproduce the problem within the last version in an isolated environment?
Thanks in advance.
Job config that shows the problem. Log from run below. Note that the workspace is /var/lib/jenkins/workspace/magjac-workspace-test, but the WORKSPACE variable is /var/lib/jenkins/workspace/saab-dev-branch_test_repo, which is another jobs workspace.
Started by user magjac
[EnvInject] - Loading node environment variables.
Building in workspace /var/lib/jenkins/workspace/magjac-workspace-test
[magjac-workspace-test] $ /bin/sh -xe /tmp/hudson102135884180944618.sh
+ env
LC_PAPER=sv_SE.UTF-8
BUILD_URL=http://worabu.netinsight.se:8080/job/magjac-workspace-test/1/
HUDSON_SERVER_COOKIE=89a7b7e616d9e722
SHELL=/bin/bash
TERM=unknown
XDG_SESSION_COOKIE=da3dd519a721d0e1b7b22e424d2ae4df-1360064288.175876-277727223
BUILD_TAG=jenkins-magjac-workspace-test-1
WORKSPACE=/var/lib/jenkins/workspace/saab-dev-branch_test_repo
JOB_URL=http://worabu.netinsight.se:8080/job/magjac-workspace-test/
USER=jenkins
LD_LIBRARY_PATH=/usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk/jre/../lib/amd64
JENKINS_HOME=/var/lib/jenkins
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/jenkins
BUILD_CAUSE_MANUALTRIGGER=true
_=/usr/bin/env
LC_COLLATE=sv_SE.UTF-8
PWD=/var/lib/jenkins/workspace/magjac-workspace-test
HUDSON_URL=http://worabu.netinsight.se:8080/
LANG=C
JOB_NAME=magjac-workspace-test
JENKINS_URL=http://worabu.netinsight.se:8080/
BUILD_CAUSE=MANUALTRIGGER
BUILD_ID=2013-02-19_08-30-44
SPEECHD_PORT=6682
HOME=/var/lib/jenkins
SHLVL=2
JENKINS_SERVER_COOKIE=89a7b7e616d9e722
EXECUTOR_NUMBER=1
NODE_LABELS=master
LOGNAME=jenkins
HUDSON_HOME=/var/lib/jenkins
LC_CTYPE=sv_SE.UTF-8
NODE_NAME=master
BUILD_NUMBER=1
HUDSON_COOKIE=d112eba8-2c55-4ff8-82d1-254f86f0b5e4
DISPLAY=:0.0
LC_TIME=en_DK.UTF-8
XAUTHORITY=/tmp/libgksu-Mbf5UN/.Xauthority
Collecting metadata...
Metadata collection done.
Finished: SUCCESS
I'm getting this issue as well. I've restarted several times, but problem didn't go away. Let me know how I can help give you more info.
[EnvInject] - Loading node environment variables.
Building on master in workspace /home/jenkins/jobs/aura-snapshot-build/workspace
Checkout:workspace / /home/jenkins/jobs/aura-snapshot-build/workspace - hudson.remoting.LocalChannel@42e934e
Using strategy: Default
Last Built Revision: Revision 1092e00a1953421e0e65447d3aa0089f933fdc92 (origin/master-pending-update-copyright-headers)
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository [OMIT]
git --version
git version 1.7.0.4
Fetching upstream changes from origin
Cleaning workspace
Resetting working tree
Seen branch in repository origin/master-pending-update-copyright-headers
Commencing build of Revision 1092e00a1953421e0e65447d3aa0089f933fdc92 (origin/master-pending-update-copyright-headers)
Checking out Revision 1092e00a1953421e0e65447d3aa0089f933fdc92 (origin/master-pending-update-copyright-headers)
Cleaning workspace
Resetting working tree
No emails were triggered.
[workspace] $ /bin/bash -xe /tmp/hudson3897671398590473161.sh
+ mkdir -p /home/jenkins/jobs/~masterArtifactMonitor/workspace/tmp
mkdir -p ${WORKSPACE}/tmp
WORKSPACE=/home/jenkins/jobs/~masterArtifactMonitor/workspace
actual workspace should be /home/jenkins/jobs/aura-snapshot-build/workspace
For now, the challenge is to reproduce this issue in an isolated environment.
For example, an environment with only a Jenkins instance and the envinject plugin.
This problem went away for about a week after rebooting a few times but just cropped up again. Any help that anyone can provide would be greatly appreciated.