We have a very large distributed Jenkins CI undertaking at the moment and our first few tests always fail due to this error.

TraderPlus.UI.Test #4641 completed : FAILURE
TraderPlus.UI.Test #4646 completed : FAILURE
Build TraderPlus.UI.Test #4655 started
Build TraderPlus.UI.Test #4654 started
TraderPlus.UI.Test #4645 completed : FAILURE
Build TraderPlus.UI.Test #4656 started
TraderPlus.UI.Test #4648 completed : FAILURE
Build TraderPlus.UI.Test #4657 started
TraderPlus.UI.Test #4639 completed : FAILURE
Build TraderPlus.UI.Test #4658 started
TraderPlus.UI.Test #4642 completed : FAILURE
Build TraderPlus.UI.Test #4659 started
TraderPlus.UI.Test #4647 completed
Build TraderPlus.UI.Test #4660 started
TraderPlus.UI.Test #4644 completed
Build TraderPlus.UI.Test #4661 started
(Rest of the tests pass)

Failure details:
ln builds\lastSuccessfulBuild C:\Users\Administrator\.jenkins\jobs\TraderPlus.UI.Test\lastSuccessful failed
java.nio.file.FileAlreadyExistsException: C:\Users\Administrator\.jenkins\jobs\TraderPlus.UI.Test\lastSuccessful
at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(Unknown Source)
at java.nio.file.Files.createSymbolicLink(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at hudson.Util.createSymlinkJava7(Util.java:1227)
at hudson.Util.createSymlink(Util.java:1143)
at hudson.model.Run.createSymlink(Run.java:1848)
at hudson.model.Run.updateSymlinks(Run.java:1829)
at hudson.model.Run.execute(Run.java:1742)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
[EnvInject] - Loading node environment variables.
Building remotely on au03-devcivm4 (UITest) in workspace D:\Jenkins\workspace\TraderPlus.UI.Test

Running Jenkins ver. 1.616

The same issue occures when two builds starts at the same time (parallel builds) on different Jenkins nodes.
Platform: Windows (master and nodes)
Jenkins: ver. 1.619
Builds run in parallel with Build Flow Plugin 0.18

In my case exceptions occurs in one of two builds (first or second, it's random) but build continues normally.

ln builds\lastSuccessfulBuild [path_to_project_workspace]\lastSuccessful failed
java.nio.file.FileAlreadyExistsException: [path_to_project_workspace]\lastSuccessful
at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(Unknown Source)
at java.nio.file.Files.createSymbolicLink(Unknown Source)
at sun.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at hudson.Util.createSymlinkJava7(Util.java:1227)
at hudson.Util.createSymlink(Util.java:1143)
at hudson.model.Run.createSymlink(Run.java:1845)
at hudson.model.Run.updateSymlinks(Run.java:1826)
at hudson.model.Run.execute(Run.java:1739)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:381)
ln builds\lastStableBuild [path_to_project_workspace]\lastStable failed
java.nio.file.FileAlreadyExistsException: [path_to_project_workspace]\lastStable
at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(Unknown Source)
at java.nio.file.Files.createSymbolicLink(Unknown Source)
at sun.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at hudson.Util.createSymlinkJava7(Util.java:1227)
at hudson.Util.createSymlink(Util.java:1143)
at hudson.model.Run.createSymlink(Run.java:1845)
at hudson.model.Run.updateSymlinks(Run.java:1827)
at hudson.model.Run.execute(Run.java:1739)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:381)

I can recreate, any updates on this?

The problem still exists in 1.643 version.

It seems the problem only occurs when the job triggered by MultiJob Plug-in. The job can run parallel if I trigger it manually.

The other thing is the problem does not occur on Windows. I can reproduce it in Linux.

I figured out, in Windows, MultiJob Plug-in does not execute the jobs in parallel. In addition, the parallel job directories (e.g. myjob@1, myjob@2, ...) were not created. This behavior explains why I do not see the problem on Windows.

In Linux, to confirm the problem is related with MultiJob Plug-in, I created a shell script to execute the job in parallel through Jenkins' Remote API:

#!/bin/sh

( curl -X POST http://jenkins.mycompany.com/jenkins/job/myjob/build?delay=0sec --user jenkinsuser:jenkinspassword ) &
( curl -X POST http://jenkins.mycompany.com/jenkins/job/myjob/build?delay=0sec --user jenkinsuser:jenkinspassword ) &
( curl -X POST http://jenkins.mycompany.com/jenkins/job/myjob/build?delay=0sec --user jenkinsuser:jenkinspassword ) &
wait
echo "completed"

The job executed in parallel and the problem did not happen. I confirmed that the parallel job directories were created successfully.

Should we changed the issue's component to multijob-plugin?

Not sure if it is really the mutli-job plugin or just the ability to start the jobs really closely together. We are experiencing this and do no have the multi-job plugin installed. In stead we use the build-flow plugin. Our use case is that we kick of multiple jobs in parallel using the parallel command in the build-flow dsl.

You seem to be right Kenneth. Thanks for the clarification.

i recently upgraded to Jenkins 1.651.1 (on windows)
we make a lot of use of the parallel builds function... running 15 jobs in parallel.
proportionately.. only a couple spit out this error... and can end up killing the entire job run.

In the same parallel run, I can end up with different fault reasons among the jobs suggesting that dependent on the timing, a couple were competing with each other. eg: access denied, file already exists....

------ job@1
16:32:42 ln builds\lastStableBuild C:\.jenkins\jobs\XXXXX\lastStable failed
16:32:42 java.nio.file.AccessDeniedException: C:\.jenkins\jobs\XXXXX\lastStable
16:32:42 at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
16:32:42 at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
16:32:42 at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
16:32:42 at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
16:32:42 at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(Unknown Source)
16:32:42 at java.nio.file.Files.deleteIfExists(Unknown Source)
16:32:42 at sun.reflect.GeneratedMethodAccessor1458.invoke(Unknown Source)
16:32:42 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
16:32:42 at java.lang.reflect.Method.invoke(Unknown Source)
16:32:42 at hudson.Util.createSymlinkJava7(Util.java:1233)
16:32:42 at hudson.Util.createSymlink(Util.java:1151)
16:32:42 at hudson.model.Run.createSymlink(Run.java:1840)
16:32:42 at hudson.model.Run.updateSymlinks(Run.java:1822)
16:32:42 at hudson.model.Run.execute(Run.java:1736)
16:32:42 at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
16:32:42 at hudson.model.ResourceController.execute(ResourceController.java:98)
16:32:42 at hudson.model.Executor.run(Executor.java:410)

------ job@2
16:32:41 ln builds\lastSuccessfulBuild C:\.jenkins\jobs\XXXX\lastSuccessful failed
16:32:41 java.nio.file.FileAlreadyExistsException: C:\.jenkins\jobs\XXXXX\lastSuccessful
16:32:41 at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
16:32:41 at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
16:32:41 at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
16:32:41 at sun.nio.fs.WindowsFileSystemProvider.createSymbolicLink(Unknown Source)
16:32:41 at java.nio.file.Files.createSymbolicLink(Unknown Source)
16:32:41 at sun.reflect.GeneratedMethodAccessor1459.invoke(Unknown Source)
16:32:41 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
16:32:41 at java.lang.reflect.Method.invoke(Unknown Source)
16:32:41 at hudson.Util.createSymlinkJava7(Util.java:1235)
16:32:41 at hudson.Util.createSymlink(Util.java:1151)
16:32:41 at hudson.model.Run.createSymlink(Run.java:1840)
16:32:41 at hudson.model.Run.updateSymlinks(Run.java:1821)
16:32:41 at hudson.model.Run.execute(Run.java:1736)
16:32:41 at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
16:32:41 at hudson.model.ResourceController.execute(ResourceController.java:98)
16:32:41 at hudson.model.Executor.run(Executor.java:410)

I was able to work around this by creating a bunch of jobs from a template job and giving each job a unique name (e.g. unit-test-1, unit-test-2).

Here's some groovy code that I put in a job to recreate the jobs. It takes a single parameter NUM_JOBS that it also updates a global jenkins environment variable called NUM_RUN_TESTS_JOBS in case your tests need to know that they are job X of NUM_RUN_TESTS_JOBS. 

import jenkins.model.Jenkins
import hudson.model.AbstractProject
def num_jobs = build.getEnvironment(listener).get("NUM_JOBS")
//update the global env property 
instance = Jenkins.getInstance()
globalNodeProperties = instance.getGlobalNodeProperties()
envVarsNodePropertyList = globalNodeProperties.getAll(hudson.slaves.EnvironmentVariablesNodeProperty.class)
envVars = envVarsNodePropertyList.get(0).getEnvVars()
envVars.put("NUM_RUN_TESTS_JOBS", num_jobs)
instance.save()
base_name = "unit-tests-"
// delete the jobs
instance.getAllItems(AbstractProject.class).each {it ->
 if(it.fullName.contains(base_name)) {
 println('deleting ' + it.fullName)
 it.delete()
 }
}
//create the jobs
template_name = "unit-test-template"
(1..num_jobs.toInteger()).each {
 copied_name = base_name + it
 println('copying ' + template_name + ' to ' + copied_name)
 try {
 def job = instance.copy(instance.getItem(template_name), copied_name);
 job.save()
 } catch (IllegalArgumentException e) {}
}