Personal builds can encounter deadlock when fetching source

Sometimes, when running personal builds from RTC, the job will hang indefinitely. The console output for the job shows that the job is in the team concert pluging and is fetching source from RTC.

Looking at the output from jenkins/ThreadDump, it would seem there is a deadlock situation occurring. Here is some example output:

Executor #1 for master : executing Mainline personal build #458

"Executor #1 for master : executing Mainline personal build #458" Id=945 Group=main TIMED_WAITING on java.util.ArrayList@5663b111
at java.lang.Object.wait(Native Method)

• waiting on java.util.ArrayList@5663b111
  at com.ibm.team.filesystem.client.internal.copyfileareas.BatchingLock.acquire(BatchingLock.java:446)
  at com.ibm.team.filesystem.client.internal.copyfileareas.CopyFileAreaManager.deregister(CopyFileAreaManager.java:350)
  at com.ibm.team.filesystem.client.internal.SharingManager.deregister(SharingManager.java:1358)
  at com.ibm.team.filesystem.client.internal.SharingManager.deregister(SharingManager.java:1316)
  at com.ibm.team.build.internal.hjplugin.rtc.RepositoryConnection.checkout(RepositoryConnection.java:457)
  at com.ibm.team.build.internal.hjplugin.rtc.RTCFacade.checkout(RTCFacade.java:390)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:622)
  at com.ibm.team.build.internal.hjplugin.RTCFacadeFactory$RTCFacadeWrapper.invoke(RTCFacadeFactory.java:115)
  at com.ibm.team.build.internal.hjplugin.RTCCheckoutTask.invoke(RTCCheckoutTask.java:166)
  at com.ibm.team.build.internal.hjplugin.RTCCheckoutTask.invoke(RTCCheckoutTask.java:32)
  at hudson.FilePath.act(FilePath.java:920)
  at hudson.FilePath.act(FilePath.java:893)
  at com.ibm.team.build.internal.hjplugin.RTCScm.checkout(RTCScm.java:1079)
  at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
  at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:615)
  at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
  at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:524)
  at hudson.model.Run.execute(Run.java:1706)
  at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
  at hudson.model.ResourceController.execute(ResourceController.java:88)
  at hudson.model.Executor.run(Executor.java:232)

I spoke with Scott Cowan over Sametime and he suggested that it was trying to access a lock on a file on the master filesystem and there might be some shared state between builds. I've had a quick look over our builds and I can't see anything obvious that would cause the problem but I will continue to investigate.

It should be noted that this only seems to occur for personal builds where a particular user's workspace is being used. Our normal builds have not seen this issue.

Thanks,
James