Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-9540

Copy Artifacts Plugin Throws "IOException: Pipe is already closed"

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • copyartifact-plugin, core
    • None
    • Windows XP SP3, Sun JRE 1.6.25

      Since updating to Jenkins 1.409 and Copy Artifacts Plugin 1.16, build jobs fail with an IOException when copying build artifacts from the Master to the current Slave.

      ERROR: Failed to copy artifacts from ABC Build with filter: _tests\UnitTests\bin\Debug**
      java.io.IOException: Pipe is already closed
      at hudson.remoting.PipeWindow.checkDeath(PipeWindow.java:83)
      at hudson.remoting.PipeWindow$Real.get(PipeWindow.java:165)
      at hudson.remoting.ProxyOutputStream._write(ProxyOutputStream.java:118)
      at hudson.remoting.ProxyOutputStream.write(ProxyOutputStream.java:103)
      at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
      at java.io.BufferedOutputStream.write(Unknown Source)
      at java.util.zip.DeflaterOutputStream.deflate(Unknown Source)
      at java.util.zip.DeflaterOutputStream.write(Unknown Source)
      at java.util.zip.GZIPOutputStream.write(Unknown Source)
      at java.io.BufferedOutputStream.write(Unknown Source)
      at org.apache.tools.tar.TarBuffer.writeBlock(TarBuffer.java:410)
      at org.apache.tools.tar.TarBuffer.writeRecord(TarBuffer.java:351)
      at hudson.org.apache.tools.tar.TarOutputStream.writeEOFRecord(TarOutputStream.java:356)
      at hudson.org.apache.tools.tar.TarOutputStream.finish(TarOutputStream.java:137)
      at hudson.org.apache.tools.tar.TarOutputStream.close(TarOutputStream.java:149)
      at hudson.util.io.TarArchiver.close(TarArchiver.java:119)
      at hudson.FilePath.writeToTar(FilePath.java:1596)
      at hudson.FilePath.copyRecursiveTo(FilePath.java:1521)
      at hudson.FilePath.copyRecursiveTo(FilePath.java:1450)
      at hudson.plugins.copyartifact.FilePathCopyMethod.copyAll(FilePathCopyMethod.java:51)
      at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:225)
      at hudson.plugins.copyartifact.CopyArtifact.perform(CopyArtifact.java:199)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649)
      at hudson.model.Build$RunnerImpl.build(Build.java:177)
      at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
      at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
      at hudson.model.Run.run(Run.java:1362)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:145)
      Caused by: java.io.IOException: Pipe is already closed
      at hudson.remoting.FastPipedOutputStream.write(FastPipedOutputStream.java:147)
      at hudson.remoting.FastPipedOutputStream.write(FastPipedOutputStream.java:131)
      at hudson.remoting.ProxyOutputStream$Chunk$1.run(ProxyOutputStream.java:185)
      at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
      at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)
      Caused by: hudson.remoting.FastPipedInputStream$ClosedBy: The pipe was closed at...
      at hudson.remoting.FastPipedInputStream.close(FastPipedInputStream.java:112)
      at hudson.FilePath$32.invoke(FilePath.java:1517)
      at hudson.FilePath$32.invoke(FilePath.java:1511)
      at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1956)
      at hudson.remoting.UserRequest.perform(UserRequest.java:118)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:270)
      at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
      at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at hudson.remoting.Engine$1$1.run(Engine.java:58)
      ... 1 more

      The Slaves are running as Windows Services under the Local System account. The slave.jar files have been updated to 1.409.

      The IOException is thrown no matter if the service is running under the Local System account, the Network Service account, under a Windows domain user account, or with desktop interaction enabled (see https://issues.jenkins-ci.org/browse/JENKINS-9456).

      If the Slave is started from a DOS prompt (java -Xrs -jar slave.jar -noCertificateCheck -jnlpUrl "https://buildserver.example.com/computer/esxi-01%20XP08/slave-agent.jnlp") the issue does not occur. The DOS prompt is started by a Windows domain user.

          [JENKINS-9540] Copy Artifacts Plugin Throws "IOException: Pipe is already closed"

          sanga added a comment -

          I found the source of my problem. A process started during the build was left running which had a lock on a file in the workspace. A couple of things about that though:

          • I thought the jenkins kept track and killed such rough processes?
          • the resulting error was quite non-intuitive. I.e. it wasn't exactly obvious (at least to me) what the actual source of the error was at first.

          sanga added a comment - I found the source of my problem. A process started during the build was left running which had a lock on a file in the workspace. A couple of things about that though: I thought the jenkins kept track and killed such rough processes? the resulting error was quite non-intuitive. I.e. it wasn't exactly obvious (at least to me) what the actual source of the error was at first.

          Eric Long added a comment -

          My errors went away once I realized that the slave was out of drive space. I cleared up space and then the "java.io.IOException: Pipe is already closed" messages disappeared and jobs completed.

          Eric Long added a comment - My errors went away once I realized that the slave was out of drive space. I cleared up space and then the "java.io.IOException: Pipe is already closed" messages disappeared and jobs completed.

          I just experienced this on my Jenkins 1.442 - and Eric's comment is what helped me. This bug should track changing that message to something related to disk space.

          Jørgen Tjernø added a comment - I just experienced this on my Jenkins 1.442 - and Eric's comment is what helped me. This bug should track changing that message to something related to disk space.

          I got this message and I don't think it is related to disk space on the slave (the disks and filesystems are huge on my machines). I'm running 1.434 on Centos master and slaves. To reproduce:
          1) Make a Job1 that gets a large* repsitoryo cloned from git (e.g. my repo: git@github.com:VoltDB/voltdb.git). Create an artifact from the whole thing (voltdb/**) Note: I assume it could be any large fileset, but I haven't tried it)
          2) Make a Job2 that copies the artifact from Job1
          Result: Job2 gets this error if I don't first delete the workspace (which I'm using a plugin to do, but really should not need to).

          I say large repo because an artifact of a subset, for example votldb/src/* works fine).

          Ruth Morgenstein added a comment - I got this message and I don't think it is related to disk space on the slave (the disks and filesystems are huge on my machines). I'm running 1.434 on Centos master and slaves. To reproduce: 1) Make a Job1 that gets a large* repsitoryo cloned from git (e.g. my repo: git@github.com:VoltDB/voltdb.git). Create an artifact from the whole thing (voltdb/**) Note: I assume it could be any large fileset, but I haven't tried it) 2) Make a Job2 that copies the artifact from Job1 Result: Job2 gets this error if I don't first delete the workspace (which I'm using a plugin to do, but really should not need to). I say large repo because an artifact of a subset, for example votldb/src/ * works fine).

          Ringo De Smet added a comment -

          I bump into this issue with Jenkins 1.445 and CopyArtifact 1.18. In my case, it is not related to a lack of diskspace on the slave. There are multiple gigabytes of free space. The workaround was to remove the work space (tnx Dirk) and trigger a clean build.

          Ringo De Smet added a comment - I bump into this issue with Jenkins 1.445 and CopyArtifact 1.18. In my case, it is not related to a lack of diskspace on the slave. There are multiple gigabytes of free space. The workaround was to remove the work space (tnx Dirk) and trigger a clean build.

          Ringo De Smet added a comment -

          Today I ended up in a situation where even the workaround of cleaning the workspace and triggering a clean build doesn't work. A new build again ends up having the exception.

          What I do notice however: I have two times the Copy Artifact as build step in place in this project. The first instance has a filter filled in, the second one not. If Copy Artifact fails, it is always the second one that fails. While I left the field "Artifacts to copy" blank to copy all artifacts, the exception has as message:

          ERROR: Failed to copy artifacts from Utilities-2.1-linux-64bit with filter: **
          

          Filling in ** as filter myself is invalid input, so where does this ** come from?

          Ringo De Smet added a comment - Today I ended up in a situation where even the workaround of cleaning the workspace and triggering a clean build doesn't work. A new build again ends up having the exception. What I do notice however: I have two times the Copy Artifact as build step in place in this project. The first instance has a filter filled in, the second one not. If Copy Artifact fails, it is always the second one that fails. While I left the field "Artifacts to copy" blank to copy all artifacts, the exception has as message: ERROR: Failed to copy artifacts from Utilities-2.1-linux-64bit with filter: ** Filling in ** as filter myself is invalid input, so where does this ** come from?

          Alan Harder added a comment -

          Alan Harder added a comment - It comes from https://github.com/jenkinsci/copyartifact-plugin/blob/master/src/main/java/hudson/plugins/copyartifact/CopyArtifact.java#L181 (ie, this is expected for empty filter).

          Hello. Is there some progress on fixing the bug? It's reproduced on our servers few times a day

          Dmitry Borovsky added a comment - Hello. Is there some progress on fixing the bug? It's reproduced on our servers few times a day

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/FilePath.java
          core/src/test/java/hudson/FilePathTest.java
          http://jenkins-ci.org/commit/jenkins/574fe1409ec80135c91054ed13afd131fa5eb48c
          Log:
          [FIXED JENKINS-9540] Failure of recursive copy on slave side should be reported.
          Instead it was being thrown in a Future that the master did not wait for:
          the writing side failed with a closed pipe exception, and this meaningless error was thrown up.
          Fixing by waiting for the reading side first, so any error there will be thrown up.
          (If there was also an error on the writing side, we would miss it, but typically this will just be the closed stream.)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/574fe1409ec80135c91054ed13afd131fa5eb48c Log: [FIXED JENKINS-9540] Failure of recursive copy on slave side should be reported. Instead it was being thrown in a Future that the master did not wait for: the writing side failed with a closed pipe exception, and this meaningless error was thrown up. Fixing by waiting for the reading side first, so any error there will be thrown up. (If there was also an error on the writing side, we would miss it, but typically this will just be the closed stream.)

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2845
          [FIXED JENKINS-9540] Failure of recursive copy on slave side should be reported. (Revision 574fe1409ec80135c91054ed13afd131fa5eb48c)

          Result = SUCCESS
          Jesse Glick : 574fe1409ec80135c91054ed13afd131fa5eb48c
          Files :

          • core/src/main/java/hudson/FilePath.java
          • changelog.html
          • core/src/test/java/hudson/FilePathTest.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2845 [FIXED JENKINS-9540] Failure of recursive copy on slave side should be reported. (Revision 574fe1409ec80135c91054ed13afd131fa5eb48c) Result = SUCCESS Jesse Glick : 574fe1409ec80135c91054ed13afd131fa5eb48c Files : core/src/main/java/hudson/FilePath.java changelog.html core/src/test/java/hudson/FilePathTest.java

            jglick Jesse Glick
            dw Dirk Weinhardt
            Votes:
            9 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated:
              Resolved: