-
Bug
-
Resolution: Fixed
-
Major
-
Windows Server 2008, NTFS, Jenkins runs as Windows Service
-
Powered by SuggestiMate
The lastStable and LastSuccessful links are created as invalid symlinks with no destination under Windows. These Files cause the backup to fail, since they are not valid and cause all sorts of zip archivers to fail.
- is duplicated by
-
JENKINS-17594 incessant jenkins.model.PeepholePermalink updateCache WARNINGS
-
- Resolved
-
- is related to
-
JENKINS-20725 maven build creates symlinkd files that Windows OS does not understand
-
- Open
-
-
JENKINS-19741 Massive jobs execution causes issues in lastSuccessfulBuild permalink update
-
- Reopened
-
-
JENKINS-18866 Symlink prevens backup from ThinBackup
-
- Closed
-
- relates to
-
JENKINS-37862 Extract build symlink handling to a plugin
-
- Resolved
-
[JENKINS-17681] LastSuccessful and LastStable symlinks are invalid under Windows
Just checked in a Jenkins dev build on 2008 R2 SP1 x64 using JDK 7u11 and the symlinks seemed to be fine from what I could see from Explorer and running dir. Send to » Compressed (zipped) folder did not work (the error message was generic). Jenkins did not report any error, either after the builds or after a restart.
Is it possible that you're running with UAC disabled? I'm using JDK 7u21 and I consistently see that stack on multiple machines, though they all run with user account control enabled.
http://superuser.com/questions/104845/permission-to-make-symbolic-links-in-windows-7 seems to suggest that creating symlinks as a normal user is not allowed.
I do not really know if I am running with UAC enabled or disabled, it is just a test installation. I know creating symlinks is only allowed under conditions on versions of Windows which support it, which is why PeepholePermalink.updateCache calls createSymlink and then checks resolveSymlink to see if it worked or not. The latter method is supposed to return null if the file is not a symlink, but it is assumed the file at least exists, and PeepholePermalink fails to check this.
I just checked on a Windows 8 machine and confirmed that the command program "mklink.exe" requires administrator privileges to create a file symlink.
The command "mklink x.jar classworlds.jar" fails with the message:
You do not have sufficient privilege to perform this operation.
If I run that same command from a window that was started with Administrator privileges, then the link is created as expected.
http://technet.microsoft.com/library/cc766301.aspx gives the explanation that creating symlinks may expose security bugs in programs not designed to handle symlinks.
In an administrator account, could reproduce the problem in a unit test. Why this does not happen when I actually run Jenkins, I am not sure, but perhaps because the test’s working directory is not on an NTFS volume. At any rate, that was enough to test my hypothesis and implement a fix; and I tried running in a regular user account and it seems to work.
Code changed in jenkins
User: Jesse Glick
Path:
core/src/main/java/jenkins/model/PeepholePermalink.java
core/src/test/java/jenkins/model/PeepholePermalinkTest.java
http://jenkins-ci.org/commit/jenkins/00d5e38759d08fa89ef366fd430e900f4138f0af
Log:
JENKINS-17681 Set up a test which passes on Unix but fails on Windows when symlinks are enabled on the platform but disabled for the current user.
Code changed in jenkins
User: Jesse Glick
Path:
changelog.html
core/src/main/java/hudson/Util.java
core/src/main/java/jenkins/model/PeepholePermalink.java
http://jenkins-ci.org/commit/jenkins/3c2b50ecfe1c58f64b32d30a3f96e647671f7d03
Log:
[FIXED JENKINS-17681] Do not try to call resolveSymlink on a nonexistent link as you will just get an IOException, breaking peephole permalinks in some Windows environments.
Integrated in jenkins_main_trunk #2505
JENKINS-17681 Set up a test which passes on Unix but fails on Windows when symlinks are enabled on the platform but disabled for the current user. (Revision 00d5e38759d08fa89ef366fd430e900f4138f0af)
[FIXED JENKINS-17681] Do not try to call resolveSymlink on a nonexistent link as you will just get an IOException, breaking peephole permalinks in some Windows environments. (Revision 3c2b50ecfe1c58f64b32d30a3f96e647671f7d03)
Result = SUCCESS
Jesse Glick : 00d5e38759d08fa89ef366fd430e900f4138f0af
Files :
- core/src/test/java/jenkins/model/PeepholePermalinkTest.java
- core/src/main/java/jenkins/model/PeepholePermalink.java
Jesse Glick : 3c2b50ecfe1c58f64b32d30a3f96e647671f7d03
Files :
- core/src/main/java/jenkins/model/PeepholePermalink.java
- core/src/main/java/hudson/Util.java
- changelog.html
Fix should be in 1.515. Marked a candidate for backporting to 1.509.2 if that is ever produced.
We are running 1.514 and have this issue. It causes tons of stack traces which slows performance. I have filtered out the messages in the logging config which helps.
+1 getting it into 1.509.2 because we want to move to LTS but won't until this is fixed
Another 1 on LTS 1.509.2. I upgraded from 1.466.1 LTS to 1.509.1 and unfortunately could not roll back. I see 10,000 lines of exceptions in my log on startup (and frequently). It could be masking other issues...
In the meantime I will run my master with Admin privileges as a workaround.
Still having zip/backup issues with the lastStable and lastSuccessful symlinks on Windows 7 with 1.515. The links are created with a forward / in the target:
05/22/2013 06:12 PM <SYMLINK> lastStable [builds/lastStableBuild] 05/22/2013 06:12 PM <SYMLINK> lastSuccessful [builds/lastSuccessfulBuild]
The forward / is incorrect, should be \ on Windows.
Code changed in jenkins
User: Jesse Glick
Path:
core/src/main/java/hudson/model/AbstractBuild.java
http://jenkins-ci.org/commit/jenkins/4af5c22134a05b6e885cbf1a721412711325c92e
Log:
[FIXED JENKINS-17681] Use builds\last
Build on Windows for permalinks.
Integrated in jenkins_main_trunk #2538
[FIXED JENKINS-17681] Use builds\last
Build on Windows for permalinks. (Revision 4af5c22134a05b6e885cbf1a721412711325c92e)
Result = SUCCESS
Jesse Glick : 4af5c22134a05b6e885cbf1a721412711325c92e
Files :
- core/src/main/java/hudson/model/AbstractBuild.java
Code changed in jenkins
User: Jesse Glick
Path:
changelog.html
http://jenkins-ci.org/commit/jenkins/334d3218ebccf7d397e8e542303bc81dbe67ca51
Log:
JENKINS-17681 Noting.
@wolfwolfswinkel please verify my latest fix, in a trunk snapshot or 1.517.
Integrated in jenkins_main_trunk #2540
JENKINS-17681 Noting. (Revision 334d3218ebccf7d397e8e542303bc81dbe67ca51)
Result = SUCCESS
Jesse Glick : 334d3218ebccf7d397e8e542303bc81dbe67ca51
Files :
- changelog.html
Using Jenkins 1.522, having issue with symlinks under windows lastFailedBuild, lastUnstableBuild & lastUnsuccessfulBuild are invalid because of this the backup fails.
@frankroghair a couple of problems were fixed here. Other problems affecting certain people may well remain, but it would be better to file them separately (with whatever details to reproduce you can muster) and link to this issue.
See issue JENKINS-20725 for an issue with the Maven Plugin that still exists in 1.509.4
This issue still exists – but running this command at an admin prompt fixes it.
fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1
We pushed this out via group policy on our domain to fix it generally, but it is annoying on non-domain machines.
Josh,
Are you seeing this for a Maven build or in other areas. Take a look at the issue JENKINS-20725 and vote that up if that is what your also experiencing.
@endquote Jenkins tries to create symlinks where it can, otherwise falling back to text files. I am not quite sure what you are reporting but it sounds like a different issue. Best to file it separately with steps to reproduce, and of course a pull request to fix it if you can manage.
@jesse @joe – My apologies, I'm pretty new to Jenkins and I don't really know how Maven fits in. I'm not specifically doing anything with Maven, but maybe it's involved in all builds?
The behavior that I'm seeing is that these symlinks can't be followed in the Windows Explorer GUI without applying that fsutil command above. Further, if I look at them from a command line, I see this:
01/09/2014 05:22 PM <SYMLINK> lastFailedBuild [-1]
01/10/2014 01:02 PM <SYMLINKD> lastStableBuild [162]
01/10/2014 01:02 PM <SYMLINKD> lastSuccessfulBuild [162]
01/05/2014 03:14 PM <SYMLINK> lastUnstableBuild [-1]
01/09/2014 05:22 PM <SYMLINK> lastUnsuccessfulBuild [-1]
A regular symlink would show the target of the link instead of a numeric code. I am trying to parse the target in a deployment script.
If I should enter this bug elsewhere, let me know. Thanks for responding.
Expanding on the above directory listing, note how the links to build numbers show the target, but the "last*" ones don't.
01/10/2014 02:25 PM <DIR> .
01/10/2014 02:25 PM <DIR> ..
01/09/2014 02:09 PM <SYMLINKD> 152 [2014-01-09_14-08-58]
01/09/2014 03:24 PM <SYMLINKD> 153 [2014-01-09_15-24-00]
01/09/2014 03:29 PM <SYMLINKD> 154 [2014-01-09_15-29-22]
01/09/2014 03:44 PM <SYMLINKD> 155 [2014-01-09_15-44-02]
01/09/2014 04:09 PM <SYMLINKD> 156 [2014-01-09_16-09-01]
01/09/2014 05:19 PM <SYMLINKD> 157 [2014-01-09_17-19-02]
01/09/2014 06:29 PM <SYMLINKD> 158 [2014-01-09_18-28-59]
01/10/2014 11:09 AM <SYMLINKD> 159 [2014-01-10_11-09-02]
01/10/2014 12:31 PM <SYMLINKD> 160 [2014-01-10_12-31-32]
01/10/2014 12:49 PM <SYMLINKD> 161 [2014-01-10_12-48-59]
01/10/2014 12:58 PM <SYMLINKD> 162 [2014-01-10_12-58-51]
01/09/2014 02:12 PM <DIR> 2014-01-09_14-08-58
01/09/2014 03:27 PM <DIR> 2014-01-09_15-24-00
01/09/2014 03:33 PM <DIR> 2014-01-09_15-29-22
01/09/2014 03:47 PM <DIR> 2014-01-09_15-44-02
01/09/2014 04:12 PM <DIR> 2014-01-09_16-09-01
01/09/2014 05:22 PM <DIR> 2014-01-09_17-19-02
01/09/2014 06:32 PM <DIR> 2014-01-09_18-28-59
01/10/2014 11:13 AM <DIR> 2014-01-10_11-09-02
01/10/2014 12:35 PM <DIR> 2014-01-10_12-31-32
01/10/2014 12:52 PM <DIR> 2014-01-10_12-48-59
01/10/2014 01:02 PM <DIR> 2014-01-10_12-58-51
01/08/2014 10:24 PM <DIR> archive
01/09/2014 05:22 PM <SYMLINK> lastFailedBuild [-1]
01/10/2014 01:02 PM <SYMLINKD> lastStableBuild [162]
01/10/2014 01:02 PM <SYMLINKD> lastSuccessfulBuild [162]
01/05/2014 03:14 PM <SYMLINK> lastUnstableBuild [-1]
01/09/2014 05:22 PM <SYMLINK> lastUnsuccessfulBuild [-1]
<SYMLINK> lastFailedBuild [-1]
<SYMLINKD> lastStableBuild [162]
looks right to me. There are no failed builds, which the bogus target -1 indicates. The last stable build is #162, so lastStableBuild links to that (which is itself a symlink to the actual build directory, 2014-01-10_12-58-51).
Whether or not the Explorer GUI allows (valid) symlinks to be traversed is a secondary consideration, so long as Jenkins code can read the symlink target.
You know, you're right. I was misinterpreting that number. This should work fine for my needs.
The issue with explorer not following the links by default is real, though.
If you can find a command which can be run as the Jenkins user which detects whether the fsutil command needs be run (as an administrator), file an RFE for Jenkins to warn you of this, most simply via a console warning but ideally using an AdministrativeMonitor. (Even better would be for Jenkins to offer to run it after you supply an administrator login, using the Windows equivalent of sudo if there is such a thing.)
This problem is also visible in Jenkins 1.509.1 LTS release candidate. It appears as a stack trace