Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24084

Slave file leak build is interrupted during Copy Artifacts build step

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core, remoting
    • Windows Server 2012 R2
      Java 1.7.0_55-b13
      Jenkins is 1.574 (slave 2.43)

      Slave processes are having file locks to files in the workspace hanging around after builds are interrupted during copy artifacts step.

      I used http://file-leak-detector.kohsuke.org/ to get a stack trace of the lock:

      File leak detector output
      #12 C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb by thread:pool-1-thread-1178 for channel on Thu Aug 21 07:28:35 CDT 2014
	at java.io.FileOutputStream.<init>(Unknown Source)
	at java.io.FileOutputStream.<init>(Unknown Source)
	at hudson.FilePath$32.call(FilePath.java:1715)
	at hudson.FilePath$32.call(FilePath.java:1710)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:328)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at hudson.remoting.Engine$1$1.run(Engine.java:63)
	at java.lang.Thread.run(Unknown Source)

      The build log for the build that corresponds to the file lock timestamp shows:

      Build console log output
      ERROR: Failed to copy artifacts from Build Empower with filter: Bin/*/Test/**/*.*,Build/**/*.*,Bin/*.runsettings
hudson.util.IOException2: Failed to copy C:\jenkins\jobs\Build Empower\builds\2014-08-20_13-38-06\archive\Bin\Release\Test\Quartz.pdb to C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb
	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:107)
	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:68)
	at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:396)
	at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:327)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:772)
	at hudson.model.Build$BuildExecution.build(Build.java:199)
	at hudson.model.Build$BuildExecution.doRun(Build.java:160)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:535)
	at hudson.model.Run.execute(Run.java:1732)
	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:234)
Caused by: java.io.InterruptedIOException
	at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:212)
	at hudson.remoting.AbstractByteArrayCommandTransport.write(AbstractByteArrayCommandTransport.java:83)
	at hudson.remoting.Channel.send(Channel.java:545)
	at hudson.remoting.ProxyOutputStream._write(ProxyOutputStream.java:163)
	at hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:109)
	at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
	at java.security.DigestOutputStream.write(Unknown Source)
	at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
	at hudson.Util.copyStream(Util.java:461)
	at hudson.FilePath$37.invoke(FilePath.java:1845)
	at hudson.FilePath$37.invoke(FilePath.java:1839)
	at hudson.FilePath.act(FilePath.java:922)
	at hudson.FilePath.act(FilePath.java:895)
	at hudson.FilePath.copyTo(FilePath.java:1839)
	at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:79)
	... 12 more
Caused by: java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:503)
	at org.jenkinsci.remoting.nio.FifoBuffer.write(FifoBuffer.java:336)
	at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:207)
	... 26 more
Build step 'Copy artifacts from another project' marked build as failure

      I'm guessing there was a network hiccup of some sort that caused the slave to momentarily lose connection with master during the copy artifacts build step.

      This is easily reproducible by simply aborting the build during this step.

          [JENKINS-24084] Slave file leak build is interrupted during Copy Artifacts build step

          mrobinet created issue -
          mrobinet made changes -
          Component/s New: copyartifact [ 15692 ]
          Description Original: Random slave processes are having file locks to files in the workspace hanging around after builds complete.

          I used http://file-leak-detector.kohsuke.org/ to get a stack trace of the lock:
          #20 C:\jenkins\workspace\Browser Tests (Smoke)\BrowserType\Firefox\OS\windows\TestCategory\BrowserRegistration\Bin\Release\Test\System.Net.Http.Extensions.xml by thread:pool-1-thread-728 for channel on Wed Jul 30 13:23:30 CDT 2014
          at java.io.FileOutputStream.<init>(Unknown Source)
          at java.io.FileOutputStream.<init>(Unknown Source)
          at hudson.FilePath$32.call(FilePath.java:1715)
          at hudson.FilePath$32.call(FilePath.java:1710)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:328)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at hudson.remoting.Engine$1$1.run(Engine.java:63)

          I can reproduce it within a few days usually, so let me know if more information is needed.           		New: Slave processes are having file locks to files in the workspace hanging around after builds are interrupted during copy artifacts step.

          I used http://file-leak-detector.kohsuke.org/ to get a stack trace of the lock:
          #12 C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb by thread:pool-1-thread-1178 for channel on Thu Aug 21 07:28:35 CDT 2014
          at java.io.FileOutputStream.<init>(Unknown Source)
          at java.io.FileOutputStream.<init>(Unknown Source)
          at hudson.FilePath$32.call(FilePath.java:1715)
          at hudson.FilePath$32.call(FilePath.java:1710)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:328)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at hudson.remoting.Engine$1$1.run(Engine.java:63)
          at java.lang.Thread.run(Unknown Source)

          The build log shows:
          ERROR: Failed to copy artifacts from Build Empower with filter: Bin/*/Test/**/*.*,Build/**/*.*,Bin/*.runsettings
          hudson.util.IOException2: Failed to copy C:\jenkins\jobs\Build Empower\builds\2014-08-20_13-38-06\archive\Bin\Release\Test\Quartz.pdb to C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:107)
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:68)
          at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:396)
          at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:327)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:772)
          at hudson.model.Build$BuildExecution.build(Build.java:199)
          at hudson.model.Build$BuildExecution.doRun(Build.java:160)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:535)
          at hudson.model.Run.execute(Run.java:1732)
          at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:234)
          Caused by: java.io.InterruptedIOException
          at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:212)
          at hudson.remoting.AbstractByteArrayCommandTransport.write(AbstractByteArrayCommandTransport.java:83)
          at hudson.remoting.Channel.send(Channel.java:545)
          at hudson.remoting.ProxyOutputStream._write(ProxyOutputStream.java:163)
          at hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:109)
          at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
          at java.security.DigestOutputStream.write(Unknown Source)
          at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
          at hudson.Util.copyStream(Util.java:461)
          at hudson.FilePath$37.invoke(FilePath.java:1845)
          at hudson.FilePath$37.invoke(FilePath.java:1839)
          at hudson.FilePath.act(FilePath.java:922)
          at hudson.FilePath.act(FilePath.java:895)
          at hudson.FilePath.copyTo(FilePath.java:1839)
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:79)
          ... 12 more
          Caused by: java.lang.InterruptedException
          at java.lang.Object.wait(Native Method)
          at java.lang.Object.wait(Object.java:503)
          at org.jenkinsci.remoting.nio.FifoBuffer.write(FifoBuffer.java:336)
          at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:207)
          ... 26 more
          Build step 'Copy artifacts from another project' marked build as failure

          I'm guessing there was a network hiccup of some sort.

          This is easily reproducible by simply aborting the build during the Copy Artifacts step.
          Summary Original: Slave file leak New: Slave file leak when interrupted builds
          mrobinet made changes -
          Summary Original: Slave file leak when interrupted builds New: Slave file leak build is interrupted during Copy Artifacts build step
          mrobinet made changes -
          Description Original: Slave processes are having file locks to files in the workspace hanging around after builds are interrupted during copy artifacts step.

          I used http://file-leak-detector.kohsuke.org/ to get a stack trace of the lock:
          #12 C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb by thread:pool-1-thread-1178 for channel on Thu Aug 21 07:28:35 CDT 2014
          at java.io.FileOutputStream.<init>(Unknown Source)
          at java.io.FileOutputStream.<init>(Unknown Source)
          at hudson.FilePath$32.call(FilePath.java:1715)
          at hudson.FilePath$32.call(FilePath.java:1710)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:328)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at hudson.remoting.Engine$1$1.run(Engine.java:63)
          at java.lang.Thread.run(Unknown Source)

          The build log shows:
          ERROR: Failed to copy artifacts from Build Empower with filter: Bin/*/Test/**/*.*,Build/**/*.*,Bin/*.runsettings
          hudson.util.IOException2: Failed to copy C:\jenkins\jobs\Build Empower\builds\2014-08-20_13-38-06\archive\Bin\Release\Test\Quartz.pdb to C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:107)
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:68)
          at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:396)
          at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:327)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:772)
          at hudson.model.Build$BuildExecution.build(Build.java:199)
          at hudson.model.Build$BuildExecution.doRun(Build.java:160)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:535)
          at hudson.model.Run.execute(Run.java:1732)
          at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:234)
          Caused by: java.io.InterruptedIOException
          at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:212)
          at hudson.remoting.AbstractByteArrayCommandTransport.write(AbstractByteArrayCommandTransport.java:83)
          at hudson.remoting.Channel.send(Channel.java:545)
          at hudson.remoting.ProxyOutputStream._write(ProxyOutputStream.java:163)
          at hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:109)
          at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
          at java.security.DigestOutputStream.write(Unknown Source)
          at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
          at hudson.Util.copyStream(Util.java:461)
          at hudson.FilePath$37.invoke(FilePath.java:1845)
          at hudson.FilePath$37.invoke(FilePath.java:1839)
          at hudson.FilePath.act(FilePath.java:922)
          at hudson.FilePath.act(FilePath.java:895)
          at hudson.FilePath.copyTo(FilePath.java:1839)
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:79)
          ... 12 more
          Caused by: java.lang.InterruptedException
          at java.lang.Object.wait(Native Method)
          at java.lang.Object.wait(Object.java:503)
          at org.jenkinsci.remoting.nio.FifoBuffer.write(FifoBuffer.java:336)
          at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:207)
          ... 26 more
          Build step 'Copy artifacts from another project' marked build as failure

          I'm guessing there was a network hiccup of some sort.

          This is easily reproducible by simply aborting the build during the Copy Artifacts step.
          		 New: Slave processes are having file locks to files in the workspace hanging around after builds are interrupted during copy artifacts step.

          I used http://file-leak-detector.kohsuke.org/ to get a stack trace of the lock:
          {code:title=File leak detector output}
          #12 C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb by thread:pool-1-thread-1178 for channel on Thu Aug 21 07:28:35 CDT 2014
          at java.io.FileOutputStream.<init>(Unknown Source)
          at java.io.FileOutputStream.<init>(Unknown Source)
          at hudson.FilePath$32.call(FilePath.java:1715)
          at hudson.FilePath$32.call(FilePath.java:1710)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:328)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at hudson.remoting.Engine$1$1.run(Engine.java:63)
          at java.lang.Thread.run(Unknown Source)
          {code}

          The build log for the build that corresponds to the file lock timestamp shows:
          {code:title=Build console log output}
          ERROR: Failed to copy artifacts from Build Empower with filter: Bin/*/Test/**/*.*,Build/**/*.*,Bin/*.runsettings
          hudson.util.IOException2: Failed to copy C:\jenkins\jobs\Build Empower\builds\2014-08-20_13-38-06\archive\Bin\Release\Test\Quartz.pdb to C:\jenkins\workspace\Browser Tests (QA)\BrowserType\InternetExplorer32\OS\windows\TestCategory\JourneyTest02\Bin\Release\Test\Quartz.pdb
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:107)
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyAll(FingerprintingCopyMethod.java:68)
          at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:396)
          at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:327)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:772)
          at hudson.model.Build$BuildExecution.build(Build.java:199)
          at hudson.model.Build$BuildExecution.doRun(Build.java:160)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:535)
          at hudson.model.Run.execute(Run.java:1732)
          at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:234)
          Caused by: java.io.InterruptedIOException
          at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:212)
          at hudson.remoting.AbstractByteArrayCommandTransport.write(AbstractByteArrayCommandTransport.java:83)
          at hudson.remoting.Channel.send(Channel.java:545)
          at hudson.remoting.ProxyOutputStream._write(ProxyOutputStream.java:163)
          at hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:109)
          at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
          at java.security.DigestOutputStream.write(Unknown Source)
          at hudson.remoting.RemoteOutputStream.write(RemoteOutputStream.java:110)
          at hudson.Util.copyStream(Util.java:461)
          at hudson.FilePath$37.invoke(FilePath.java:1845)
          at hudson.FilePath$37.invoke(FilePath.java:1839)
          at hudson.FilePath.act(FilePath.java:922)
          at hudson.FilePath.act(FilePath.java:895)
          at hudson.FilePath.copyTo(FilePath.java:1839)
          at hudson.plugins.copyartifact.FingerprintingCopyMethod.copyOne(FingerprintingCopyMethod.java:79)
          ... 12 more
          Caused by: java.lang.InterruptedException
          at java.lang.Object.wait(Native Method)
          at java.lang.Object.wait(Object.java:503)
          at org.jenkinsci.remoting.nio.FifoBuffer.write(FifoBuffer.java:336)
          at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.writeBlock(NioChannelHub.java:207)
          ... 26 more
          Build step 'Copy artifacts from another project' marked build as failure
          {code}

          I'm guessing there was a network hiccup of some sort that caused the slave to momentarily lose connection with master during the copy artifacts build step.

          This is easily reproducible by simply aborting the build during this step.
          Oleg Nenashev made changes -
          Labels New: remoting
          ikedam made changes -
          Environment Original: Windows Server 2012 R2
          Java 1.7.0_55-b13           		New: Windows Server 2012 R2
          Java 1.7.0_55-b13
          Jenkins is 1.574 (slave 2.43)
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 156957 ] New: JNJira + In-Review [ 179458 ]
          Oleg Nenashev made changes -
          Assignee New: Oleg Nenashev [ oleg_nenashev ]
          Oleg Nenashev made changes -
          Component/s New: remoting [ 15489 ]
          Component/s Original: copyartifact-plugin [ 15692 ]
          Oleg Nenashev made changes -
          Assignee Original: Oleg Nenashev [ oleg_nenashev ]

            Unassigned Unassigned
            mrobinet mrobinet
            Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: