Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20769

Failed to execute command Pipe.EOF

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Errors from log, occurs from time to time:

      Nov 26, 2013 4:31:58 PM hudson.remoting.Channel$2 handle
      SEVERE: Failed to execute command Pipe.EOF(49) (channel qa3)
      java.lang.NullPointerException
              at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
              at hudson.remoting.Channel$2.handle(Channel.java:461)
              at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
      Nov 26, 2013 4:31:58 PM hudson.remoting.Channel$2 handle
      SEVERE: This command is created here
      Command Pipe.EOF(49) created at
              at hudson.remoting.Command.<init>(Command.java:67)
              at hudson.remoting.Command.<init>(Command.java:50)
              at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169)
              at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
              at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
              at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
              at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
              at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
              at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
      

        Attachments

          Issue Links

            Activity

            rprots Roman Prots' created issue -
            Hide
            mika Michael Prokop added a comment -

            Noticing the same here (Jenkins v1.541, using swarm plugin):

            Dec 09, 2013 4:09:54 PM SEVERE hudson.remoting.Channel$2 handle

            Failed to execute command Pipe.EOF(2053) (channel selenium-client1)
            java.lang.NullPointerException
            at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
            at hudson.remoting.Channel$2.handle(Channel.java:461)
            at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)

            Dec 09, 2013 4:09:54 PM SEVERE hudson.remoting.Channel$2 handle

            This command is created here
            Command Pipe.EOF(2053) created at
            at hudson.remoting.Command.(Command.java:62)
            at hudson.remoting.Command.(Command.java:47)
            at hudson.remoting.ProxyWriter$EOF.(ProxyWriter.java:169)
            at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
            at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
            at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
            at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
            at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
            at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)

            Show
            mika Michael Prokop added a comment - Noticing the same here (Jenkins v1.541, using swarm plugin): Dec 09, 2013 4:09:54 PM SEVERE hudson.remoting.Channel$2 handle Failed to execute command Pipe.EOF(2053) (channel selenium-client1) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Dec 09, 2013 4:09:54 PM SEVERE hudson.remoting.Channel$2 handle This command is created here Command Pipe.EOF(2053) created at at hudson.remoting.Command.(Command.java:62) at hudson.remoting.Command.(Command.java:47) at hudson.remoting.ProxyWriter$EOF.(ProxyWriter.java:169) at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124) at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Labels remoting
            Hide
            jglick Jesse Glick added a comment -

            I committed https://github.com/jenkinsci/remoting/commit/622af099d6b94569c873b8fcd2d1bc65860a8cc6 but am not sure of the root cause here. Possibly a race condition while trying to shut down a remoting channel.

            Show
            jglick Jesse Glick added a comment - I committed https://github.com/jenkinsci/remoting/commit/622af099d6b94569c873b8fcd2d1bc65860a8cc6 but am not sure of the root cause here. Possibly a race condition while trying to shut down a remoting channel.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/hudson/remoting/ProxyInputStream.java
            src/main/java/hudson/remoting/ProxyOutputStream.java
            http://jenkins-ci.org/commit/remoting/eb3a4c380e695580174ce381a8330e8e2b8a9b2e
            Log:
            JENKINS-20769 Trying to fix/suppress some related observed or actual NPEs.
            java.lang.NullPointerException
            at hudson.remoting.ProxyOutputStream$Flush$1.run(ProxyOutputStream.java:305)
            at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
            at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
            at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/remoting/ProxyInputStream.java src/main/java/hudson/remoting/ProxyOutputStream.java http://jenkins-ci.org/commit/remoting/eb3a4c380e695580174ce381a8330e8e2b8a9b2e Log: JENKINS-20769 Trying to fix/suppress some related observed or actual NPEs. java.lang.NullPointerException at hudson.remoting.ProxyOutputStream$Flush$1.run(ProxyOutputStream.java:305) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
            Hide
            aheritier Arnaud Héritier added a comment - - edited

            I have it also with 1.532.1 and 1.532.2-SNAPSHOT (rc-01/13/2014 23:54 GMT-kohsuke)

            Show
            aheritier Arnaud Héritier added a comment - - edited I have it also with 1.532.1 and 1.532.2-SNAPSHOT (rc-01/13/2014 23:54 GMT-kohsuke)
            Hide
            vmassol Vincent Massol added a comment -

            I'm also seeing this in 1.549:

            Feb 4, 2014 9:10:08 AM com.cloudbees.jenkins.GitHubPushTrigger$1 run
            INFO: SCM changes detected in xwiki-platform. Triggering  #4954
            Feb 4, 2014 9:10:28 AM hudson.remoting.Channel$2 handle
            SEVERE: Failed to execute command Pipe.EOF(9) (channel agent-2-2)
            java.lang.NullPointerException
                    at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
                    at hudson.remoting.Channel$2.handle(Channel.java:461)
                    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)Feb 4, 2014 9:10:28 AM hudson.remoting.Channel$2 handle
            SEVERE: This command is created here
            Command Pipe.EOF(9) created at        at hudson.remoting.Command.<init>(Command.java:67)
                    at hudson.remoting.Command.<init>(Command.java:50)
                    at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169)        at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
                    at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
                    at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
                    at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
                    at java.lang.ref.Finalizer.access$100(Finalizer.java:14)        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172)
            Feb 4, 2014 9:10:31 AM hudson.remoting.Channel$2 handle
            SEVERE: Failed to execute command Pipe.EOF(9) (channel agent-2-3)
            java.lang.NullPointerException
                    at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
                    at hudson.remoting.Channel$2.handle(Channel.java:461)
                    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            Feb 4, 2014 9:10:31 AM hudson.remoting.Channel$2 handle
            SEVERE: This command is created here
            Command Pipe.EOF(9) created at
                    at hudson.remoting.Command.<init>(Command.java:67)
                    at hudson.remoting.Command.<init>(Command.java:50)
                    at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169)
                    at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
                    at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
                    at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
                    at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
                    at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
            ...
            

            Is it a problem? What can I do?

            Show
            vmassol Vincent Massol added a comment - I'm also seeing this in 1.549: Feb 4, 2014 9:10:08 AM com.cloudbees.jenkins.GitHubPushTrigger$1 run INFO: SCM changes detected in xwiki-platform. Triggering #4954 Feb 4, 2014 9:10:28 AM hudson.remoting.Channel$2 handle SEVERE: Failed to execute command Pipe.EOF(9) (channel agent-2-2) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)Feb 4, 2014 9:10:28 AM hudson.remoting.Channel$2 handle SEVERE: This command is created here Command Pipe.EOF(9) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169) at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124) at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172) Feb 4, 2014 9:10:31 AM hudson.remoting.Channel$2 handle SEVERE: Failed to execute command Pipe.EOF(9) (channel agent-2-3) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Feb 4, 2014 9:10:31 AM hudson.remoting.Channel$2 handle SEVERE: This command is created here Command Pipe.EOF(9) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169) at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124) at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) ... Is it a problem? What can I do?
            Hide
            zzzeek mike bayer added a comment -

            I've recently upgraded from fedora 14 to fedora 20 and am suddenly getting this one quite often. for me it seems to occur after it provisioned a new EC2 node.

            Mar 04, 2014 8:03:49 AM hudson.slaves.NodeProvisioner update
            INFO: pg, mysql, etc (ami-d53f32bc) provisioningE successfully completed. We have now 2 computer(s)
            Mar 04, 2014 8:04:09 AM hudson.remoting.Channel$2 handle
            SEVERE: Failed to execute command Pipe.EOF(10) (channel pg, mysql, etc (i-e5638cc6))
            java.lang.NullPointerException
            	at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
            	at hudson.remoting.Channel$2.handle(Channel.java:461)
            	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            
            
            Show
            zzzeek mike bayer added a comment - I've recently upgraded from fedora 14 to fedora 20 and am suddenly getting this one quite often. for me it seems to occur after it provisioned a new EC2 node. Mar 04, 2014 8:03:49 AM hudson.slaves.NodeProvisioner update INFO: pg, mysql, etc (ami-d53f32bc) provisioningE successfully completed. We have now 2 computer(s) Mar 04, 2014 8:04:09 AM hudson.remoting.Channel$2 handle SEVERE: Failed to execute command Pipe.EOF(10) (channel pg, mysql, etc (i-e5638cc6)) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            Hide
            sag47 Sam Gleske added a comment - - edited

            I have this issue though I may be able to provide a more reliable repeatable method. My setup is as follows.

            -> Jenkins running in JBoss as jboss system user (RHEL 6.5) - jboss-eap-5.1.2 - Jenkins ver. 1.532.2
              -> slave.jar running on RHEL6.5 slave as jenkins system user which is started via SSH keys.
            -> Eclipse Standard 4.3.2 - eclipse-standard-kepler-SR2-win32-x86_64
              -> Tried Mylyn build connector for Jenkins using both latest or weekly
              -> Execute a build from the Builds view.
            

            When I run a build a couple of times every once in a while the JBoss server will crash completely. Here's the log output.

            13:34:33,581 SEVERE [Channel] Failed to execute command Pipe.EOF(22) (channel <host omitted>)
            java.lang.NullPointerException
                    at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
                    at hudson.remoting.Channel$2.handle(Channel.java:461)
                    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            13:34:33,581 SEVERE [Channel] This command is created here
            Command Pipe.EOF(22) created at
                    at hudson.remoting.Command.<init>(Command.java:67)
                    at hudson.remoting.Command.<init>(Command.java:50)
                    at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169)
                    at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
                    at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
                    at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
                    at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
                    at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
                    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
            13:36:39,627 INFO  [Run] Coeus-Local #31 main build action completed: SUCCESS
            /app/jboss/bin/run.sh: line 284: 20872 Killed                  "/app/java/bin/java" -Dprogram.name=run.sh -server -Xms1303m -Xmx1303m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.lang.ClassLoader.allowArraySyntax=true -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs="/app/jboss/lib/endorsed" -classpath "/app/jboss/bin/run.jar:/app/java/lib/tools.jar" org.jboss.Main -c jenkins-ci -b <interface omitted>
            

            In Eclipse, I get errors like the following. The Eclipse error happens every time but the Jenkins/JBoss crash doesn't always happen.

            Message:
            Unexpected error: Unexpected response from Hudson server for '/jenkins/job/Coeus-Local/build': Created
            
            Exception Stack Trace:
            org.eclipse.mylyn.internal.hudson.core.client.HudsonException: Unexpected response from Hudson server for '/jenkins/job/Coeus-Local/build': Created
            	at org.eclipse.mylyn.internal.hudson.core.client.HudsonOperation.validate(HudsonOperation.java:235)
            	at org.eclipse.mylyn.internal.hudson.core.client.RestfulHudsonClient$7.doValidate(RestfulHudsonClient.java:359)
            	at org.eclipse.mylyn.internal.hudson.core.client.HudsonOperation.processAndRelease(HudsonOperation.java:220)
            	at org.eclipse.mylyn.internal.hudson.core.client.RestfulHudsonClient$7.execute(RestfulHudsonClient.java:353)
            	at org.eclipse.mylyn.internal.hudson.core.client.HudsonOperation.run(HudsonOperation.java:167)
            	at org.eclipse.mylyn.internal.hudson.core.client.RestfulHudsonClient.runBuild(RestfulHudsonClient.java:361)
            	at org.eclipse.mylyn.internal.hudson.core.HudsonServerBehaviour.runBuild(HudsonServerBehaviour.java:791)
            	at org.eclipse.mylyn.builds.internal.core.operations.RunBuildOperation$1.doExecute(RunBuildOperation.java:55)
            	at org.eclipse.mylyn.builds.internal.core.operations.BuildJob.run(BuildJob.java:79)
            	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
            
            Session Data:
            eclipse.buildId=4.3.2.M20140221-1700
            java.version=1.7.0_45
            java.vendor=Oracle Corporation
            BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=en_US
            Framework arguments:  -product org.eclipse.epp.package.standard.product
            Command-line arguments:  -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp.package.standard.product
            

            I don't know what the plugin is doing over the API to cause this but since it works sometimes but not others I would agree it is likely a race condition.

            Show
            sag47 Sam Gleske added a comment - - edited I have this issue though I may be able to provide a more reliable repeatable method. My setup is as follows. -> Jenkins running in JBoss as jboss system user (RHEL 6.5) - jboss-eap-5.1.2 - Jenkins ver. 1.532.2 -> slave.jar running on RHEL6.5 slave as jenkins system user which is started via SSH keys. -> Eclipse Standard 4.3.2 - eclipse-standard-kepler-SR2-win32-x86_64 -> Tried Mylyn build connector for Jenkins using both latest or weekly -> Execute a build from the Builds view. When I run a build a couple of times every once in a while the JBoss server will crash completely. Here's the log output. 13:34:33,581 SEVERE [Channel] Failed to execute command Pipe.EOF(22) (channel <host omitted>) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) 13:34:33,581 SEVERE [Channel] This command is created here Command Pipe.EOF(22) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169) at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124) at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160) 13:36:39,627 INFO [Run] Coeus-Local #31 main build action completed: SUCCESS /app/jboss/bin/run.sh: line 284: 20872 Killed "/app/java/bin/java" -Dprogram.name=run.sh -server -Xms1303m -Xmx1303m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning= true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.lang. ClassLoader .allowArraySyntax= true -Djava.awt.headless= true -Djava.net.preferIPv4Stack= true -Djava.endorsed.dirs= "/app/jboss/lib/endorsed" -classpath "/app/jboss/bin/run.jar:/app/java/lib/tools.jar" org.jboss.Main -c jenkins-ci -b < interface omitted> In Eclipse, I get errors like the following. The Eclipse error happens every time but the Jenkins/JBoss crash doesn't always happen. Message: Unexpected error: Unexpected response from Hudson server for '/jenkins/job/Coeus-Local/build' : Created Exception Stack Trace: org.eclipse.mylyn.internal.hudson.core.client.HudsonException: Unexpected response from Hudson server for '/jenkins/job/Coeus-Local/build' : Created at org.eclipse.mylyn.internal.hudson.core.client.HudsonOperation.validate(HudsonOperation.java:235) at org.eclipse.mylyn.internal.hudson.core.client.RestfulHudsonClient$7.doValidate(RestfulHudsonClient.java:359) at org.eclipse.mylyn.internal.hudson.core.client.HudsonOperation.processAndRelease(HudsonOperation.java:220) at org.eclipse.mylyn.internal.hudson.core.client.RestfulHudsonClient$7.execute(RestfulHudsonClient.java:353) at org.eclipse.mylyn.internal.hudson.core.client.HudsonOperation.run(HudsonOperation.java:167) at org.eclipse.mylyn.internal.hudson.core.client.RestfulHudsonClient.runBuild(RestfulHudsonClient.java:361) at org.eclipse.mylyn.internal.hudson.core.HudsonServerBehaviour.runBuild(HudsonServerBehaviour.java:791) at org.eclipse.mylyn.builds.internal.core.operations.RunBuildOperation$1.doExecute(RunBuildOperation.java:55) at org.eclipse.mylyn.builds.internal.core.operations.BuildJob.run(BuildJob.java:79) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53) Session Data: eclipse.buildId=4.3.2.M20140221-1700 java.version=1.7.0_45 java.vendor=Oracle Corporation BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=en_US Framework arguments: -product org.eclipse.epp. package .standard.product Command-line arguments: -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp. package .standard.product I don't know what the plugin is doing over the API to cause this but since it works sometimes but not others I would agree it is likely a race condition.
            Hide
            sag47 Sam Gleske added a comment -

            Regarding my previous comment. It was not because of a race condition. It was due to my VM being out of memory. I increased the VM memory from 2GB -> 16GB and it was resolved.

            Show
            sag47 Sam Gleske added a comment - Regarding my previous comment. It was not because of a race condition. It was due to my VM being out of memory. I increased the VM memory from 2GB -> 16GB and it was resolved.
            Hide
            samxiao Sam Xiao added a comment -

            It happens to me as well:

            • Jenkins ver. 1.532.2
            • Ubuntu 12.04
            SEVERE: Failed to execute command Pipe.EOF(165) (channel bob-10.69)
            java.lang.NullPointerException
            	at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
            	at hudson.remoting.Channel$2.handle(Channel.java:461)
            	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            
            Mar 26, 2014 1:04:48 PM hudson.remoting.Channel$2 handle
            SEVERE: This command is created here
            Command Pipe.EOF(165) created at
            	at hudson.remoting.Command.<init>(Command.java:67)
            	at hudson.remoting.Command.<init>(Command.java:50)
            	at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169)
            	at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
            	at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
            	at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
            	at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
            	at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
            	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
            
            Mar 26, 2014 1:04:48 PM hudson.remoting.Channel$2 handle
            SEVERE: Failed to execute command Pipe.EOF(386) (channel bob-10.16)
            java.lang.NullPointerException
            	at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177)
            	at hudson.remoting.Channel$2.handle(Channel.java:461)
            	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            
            Mar 26, 2014 1:04:48 PM hudson.remoting.Channel$2 handle
            SEVERE: This command is created here
            Command Pipe.EOF(386) created at
            	at hudson.remoting.Command.<init>(Command.java:67)
            	at hudson.remoting.Command.<init>(Command.java:50)
            	at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169)
            	at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124)
            	at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132)
            	at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
            	at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
            	at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
            	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
            
            
            Show
            samxiao Sam Xiao added a comment - It happens to me as well: Jenkins ver. 1.532.2 Ubuntu 12.04 SEVERE: Failed to execute command Pipe.EOF(165) (channel bob-10.69) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Mar 26, 2014 1:04:48 PM hudson.remoting.Channel$2 handle SEVERE: This command is created here Command Pipe.EOF(165) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169) at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124) at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101) at java.lang.ref.Finalizer.access$100(Finalizer.java:32) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190) Mar 26, 2014 1:04:48 PM hudson.remoting.Channel$2 handle SEVERE: Failed to execute command Pipe.EOF(386) (channel bob-10.16) java.lang.NullPointerException at hudson.remoting.ProxyWriter$EOF.execute(ProxyWriter.java:177) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Mar 26, 2014 1:04:48 PM hudson.remoting.Channel$2 handle SEVERE: This command is created here Command Pipe.EOF(386) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyWriter$EOF.<init>(ProxyWriter.java:169) at hudson.remoting.ProxyWriter.close(ProxyWriter.java:124) at hudson.remoting.ProxyWriter.finalize(ProxyWriter.java:132) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101) at java.lang.ref.Finalizer.access$100(Finalizer.java:32) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            I found it interesting that this problem appears to be only seen with ProxyWriter, while more or less equivalent and more prevelant ProxyOutputStream do not show up in the stack.

            Show
            kohsuke Kohsuke Kawaguchi added a comment - I found it interesting that this problem appears to be only seen with ProxyWriter , while more or less equivalent and more prevelant ProxyOutputStream do not show up in the stack.
            Hide
            dimitkoto Dimitar Sakarov added a comment -

            +1

            I am also experiencing the same issue with the ProxyWriter only mentioned in the stack, the same way as the ones in the earlier comments.

            Show
            dimitkoto Dimitar Sakarov added a comment - +1 I am also experiencing the same issue with the ProxyWriter only mentioned in the stack, the same way as the ones in the earlier comments.
            Hide
            jglick Jesse Glick added a comment -

            @kohsuke as noted in my eb3a4c3 commit comment I have observed this also in ProxyOutputStream.

            Show
            jglick Jesse Glick added a comment - @kohsuke as noted in my eb3a4c3 commit comment I have observed this also in ProxyOutputStream .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/ProxyInputStream.java
            src/main/java/hudson/remoting/ProxyOutputStream.java
            http://jenkins-ci.org/commit/remoting/ec1a7c8e982d6cad5cbdd3d4b9ca3c06404f037c
            Log:
            Revert "JENKINS-20769 Trying to fix/suppress some related observed or actual NPEs."

            This reverts commit eb3a4c380e695580174ce381a8330e8e2b8a9b2e.

            Reverting this commit to unmask the problem.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/ProxyInputStream.java src/main/java/hudson/remoting/ProxyOutputStream.java http://jenkins-ci.org/commit/remoting/ec1a7c8e982d6cad5cbdd3d4b9ca3c06404f037c Log: Revert " JENKINS-20769 Trying to fix/suppress some related observed or actual NPEs." This reverts commit eb3a4c380e695580174ce381a8330e8e2b8a9b2e. Reverting this commit to unmask the problem.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/ProxyWriter.java
            http://jenkins-ci.org/commit/remoting/d1e0fe07d8f72d37d9064404f94f7ae3bd6b223a
            Log:
            Revert "NPE observed in log while trying to reconnect a slave."

            This reverts commit 622af099d6b94569c873b8fcd2d1bc65860a8cc6.

            Simiarly, reverting to unmask the root cause of JENKINS-20769.
            These problems are indicative of a deeper problem, and shouldn't
            be just swept under the rug.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/ProxyWriter.java http://jenkins-ci.org/commit/remoting/d1e0fe07d8f72d37d9064404f94f7ae3bd6b223a Log: Revert "NPE observed in log while trying to reconnect a slave." This reverts commit 622af099d6b94569c873b8fcd2d1bc65860a8cc6. Simiarly, reverting to unmask the root cause of JENKINS-20769 . These problems are indicative of a deeper problem, and shouldn't be just swept under the rug.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/test/java/hudson/remoting/ProxyWriterTest.java
            http://jenkins-ci.org/commit/remoting/e498f5aa99af88d5d2e621fbb1e85b2d6306e61f
            Log:
            JENKINS-20769 adding a test case that reproduces the problem.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/test/java/hudson/remoting/ProxyWriterTest.java http://jenkins-ci.org/commit/remoting/e498f5aa99af88d5d2e621fbb1e85b2d6306e61f Log: JENKINS-20769 adding a test case that reproduces the problem.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/ProxyOutputStream.java
            src/main/java/hudson/remoting/ProxyWriter.java
            http://jenkins-ci.org/commit/remoting/96c670de534de269bcde729a932655842a3aea2a
            Log:
            [FIXED JENKINS-20769]

            When ProxyWriter is finalized, it should attempt to release a remote reference count,
            but it shouldn't try to close the stream. For one, the other side that had exported
            the writer might be still using it (imagine exporting System.out), and secondly, the
            said object could have already been unexported, for example if the closure that brought
            ProxyWriter to this side has already returned, since RemoteWriter is auto-unexporting.

            Compare ProxyOutputStream impl vs ProxyWriter impl and the difference becomes clearer.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/ProxyOutputStream.java src/main/java/hudson/remoting/ProxyWriter.java http://jenkins-ci.org/commit/remoting/96c670de534de269bcde729a932655842a3aea2a Log: [FIXED JENKINS-20769] When ProxyWriter is finalized, it should attempt to release a remote reference count, but it shouldn't try to close the stream. For one, the other side that had exported the writer might be still using it (imagine exporting System.out), and secondly, the said object could have already been unexported, for example if the closure that brought ProxyWriter to this side has already returned, since RemoteWriter is auto-unexporting. Compare ProxyOutputStream impl vs ProxyWriter impl and the difference becomes clearer.
            scm_issue_link SCM/JIRA link daemon made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/ProxyWriter.java
            http://jenkins-ci.org/commit/remoting/57c3f1ab9e866ae237a568bd57382b6fe900ab98
            Log:
            JENKINS-20769

            Unexport is new, so for older versions fall back to EOF.
            Better to have a bogus error message than memory leak.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/ProxyWriter.java http://jenkins-ci.org/commit/remoting/57c3f1ab9e866ae237a568bd57382b6fe900ab98 Log: JENKINS-20769 Unexport is new, so for older versions fall back to EOF. Better to have a bogus error message than memory leak.
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            I have fixed this problem with ProxyWriter. As noted in my comment of 96c670de534de269bcde729a932655842a3aea2a, the problem is that this EOF command was bogus. ProxyOutputStream handles this correctly, which explains why we see this on ProxyWriter but not on ProxyOutputStream (or at least less of, given Jesse Glick comment above that he has seen it with ProxyOutputStream.

            Show
            kohsuke Kohsuke Kawaguchi added a comment - I have fixed this problem with ProxyWriter . As noted in my comment of 96c670de534de269bcde729a932655842a3aea2a, the problem is that this EOF command was bogus. ProxyOutputStream handles this correctly, which explains why we see this on ProxyWriter but not on ProxyOutputStream (or at least less of, given Jesse Glick comment above that he has seen it with ProxyOutputStream .
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            The stack trace Jesse cites in eb3a4c3 is the following, and this is a very different pattern. This is not from EOF but for Flush, and it wasn't induced by a finalizer. This one smells a genuine reference counting error, and should be tracked separately:

                java.lang.NullPointerException
                    at hudson.remoting.ProxyOutputStream$Flush$1.run(ProxyOutputStream.java:305)
                    at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
                    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
                    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
                    at java.util.concurrent.FutureTask.run(Unknown Source)
                    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
            
            Show
            kohsuke Kohsuke Kawaguchi added a comment - The stack trace Jesse cites in eb3a4c3 is the following, and this is a very different pattern. This is not from EOF but for Flush , and it wasn't induced by a finalizer. This one smells a genuine reference counting error, and should be tracked separately: java.lang.NullPointerException at hudson.remoting.ProxyOutputStream$Flush$1.run(ProxyOutputStream.java:305) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            changelog.html
            pom.xml
            http://jenkins-ci.org/commit/jenkins/11458c956e64673d99a9dc2c05cfd5f9533b4e1b
            Log:
            [JENKINS-20769 JENKINS-20204] integrated remoting 2.35 in the core.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/11458c956e64673d99a9dc2c05cfd5f9533b4e1b Log: [JENKINS-20769 JENKINS-20204] integrated remoting 2.35 in the core.
            Hide
            zzzeek mike bayer added a comment -

            just to update, I'm running "Jenkins ver. 1.558-SNAPSHOT (rc-04/01/2014 04:22 GMT-kohsuke)" which I assume includes this fix, I now see this in my logs:

            SEVERE: Failed to execute command Pipe.Flush(10) (channel general (i-34cf7b17))
            java.lang.ClassCastException: java.io.OutputStreamWriter cannot be cast to java.io.OutputStream
                    at hudson.remoting.ProxyWriter$Flush.execute(ProxyWriter.java:285)
                    at hudson.remoting.Channel$2.handle(Channel.java:461)
                    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            
            Apr 01, 2014 3:55:54 PM hudson.remoting.Channel$2 handle
            SEVERE: This command is created here
            Apr 01, 2014 3:57:13 PM hudson.remoting.Channel$2 handle
            SEVERE: Failed to execute command Pipe.Flush(10) (channel general (i-6ac87c49))
            java.lang.ClassCastException: java.io.OutputStreamWriter cannot be cast to java.io.OutputStream
                    at hudson.remoting.ProxyWriter$Flush.execute(ProxyWriter.java:285)
                    at hudson.remoting.Channel$2.handle(Channel.java:461)
                    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
            
            

            not sure if this is related to the fix or not.

            Show
            zzzeek mike bayer added a comment - just to update, I'm running "Jenkins ver. 1.558-SNAPSHOT (rc-04/01/2014 04:22 GMT-kohsuke)" which I assume includes this fix, I now see this in my logs: SEVERE: Failed to execute command Pipe.Flush(10) (channel general (i-34cf7b17)) java.lang.ClassCastException: java.io.OutputStreamWriter cannot be cast to java.io.OutputStream at hudson.remoting.ProxyWriter$Flush.execute(ProxyWriter.java:285) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Apr 01, 2014 3:55:54 PM hudson.remoting.Channel$2 handle SEVERE: This command is created here Apr 01, 2014 3:57:13 PM hudson.remoting.Channel$2 handle SEVERE: Failed to execute command Pipe.Flush(10) (channel general (i-6ac87c49)) java.lang.ClassCastException: java.io.OutputStreamWriter cannot be cast to java.io.OutputStream at hudson.remoting.ProxyWriter$Flush.execute(ProxyWriter.java:285) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) not sure if this is related to the fix or not.
            Show
            jglick Jesse Glick added a comment - @zzzeek added a note to @kohsuke’s commit: https://github.com/jenkinsci/remoting/commit/cdbc40b8980d3ea528473daf7e44475c956a9a03#commitcomment-5870985
            jglick Jesse Glick made changes -
            Assignee Kohsuke Kawaguchi [ kohsuke ]
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Hide
            olivergondza Oliver Gondža added a comment -

            Resolved integrating remoting:2.36 in 75447b57b655e38fdb48f1e854a4b287071342cf.

            Show
            olivergondza Oliver Gondža added a comment - Resolved integrating remoting:2.36 in 75447b57b655e38fdb48f1e854a4b287071342cf.
            olivergondza Oliver Gondža made changes -
            Resolution Fixed [ 1 ]
            Status Reopened [ 4 ] Resolved [ 5 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            pom.xml
            http://jenkins-ci.org/commit/jenkins/a89aa713b2ba5c325cd296d5733a565aa5cc63ec
            Log:
            [JENKINS-20769 JENKINS-20204] Integrating remoting 2.36

            Fixed a bug Jesse found in remoting

            (cherry picked from commit 75447b57b655e38fdb48f1e854a4b287071342cf)

            Conflicts:
            pom.xml

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: pom.xml http://jenkins-ci.org/commit/jenkins/a89aa713b2ba5c325cd296d5733a565aa5cc63ec Log: [JENKINS-20769 JENKINS-20204] Integrating remoting 2.36 Fixed a bug Jesse found in remoting (cherry picked from commit 75447b57b655e38fdb48f1e854a4b287071342cf) Conflicts: pom.xml
            olivergondza Oliver Gondža made changes -
            Labels remoting 1.554.1-fixed remoting
            jsirex jsirex made changes -
            Link This issue is related to JENKINS-22734 [ JENKINS-22734 ]
            Hide
            ivaylo_bratoev Ivaylo Bratoev added a comment - - edited

            This will make it in the next LTS version, right?
            Disregard - I found out what 1.554.1-fixed means Awaiting next LTS build release.

            Show
            ivaylo_bratoev Ivaylo Bratoev added a comment - - edited This will make it in the next LTS version, right? Disregard - I found out what 1.554.1-fixed means Awaiting next LTS build release.
            ivaylo_bratoev Ivaylo Bratoev made changes -
            Labels 1.554.1-fixed remoting 1.554.1-fixed lts-candidate remoting
            ivaylo_bratoev Ivaylo Bratoev made changes -
            Labels 1.554.1-fixed lts-candidate remoting 1.554.1-fixed remoting
            Hide
            baztheman baz theman added a comment -

            Which version of jenkins includes the fix(es)?

            Show
            baztheman baz theman added a comment - Which version of jenkins includes the fix(es)?
            Hide
            aje Andrew Erickson added a comment - - edited

            From scanning the changelog, 1.558 and later have the fix. LTS does not have it (not sure if it needs to be labeled lts-candidate to be picked up or not).

            Update: added the label

            Show
            aje Andrew Erickson added a comment - - edited From scanning the changelog, 1.558 and later have the fix. LTS does not have it (not sure if it needs to be labeled lts-candidate to be picked up or not). Update: added the label
            aje Andrew Erickson made changes -
            Labels 1.554.1-fixed remoting 1.554.1-fixed lts-candidate remoting
            Hide
            aje Andrew Erickson added a comment -

            Was this really fixed in 1.554.1 (that's what the label means per https://wiki.jenkins-ci.org/display/JENKINS/LTS+Release+Line)? The changelog doesn't mention it.

            Show
            aje Andrew Erickson added a comment - Was this really fixed in 1.554.1 (that's what the label means per https://wiki.jenkins-ci.org/display/JENKINS/LTS+Release+Line)? The changelog doesn't mention it.
            Hide
            olivergondza Oliver Gondža added a comment -

            Andrew Erickson: The LTS changelogs are not reliable in this regard. You are right to say that fix went into 1.554.1. See cloudbees changlog that tends to be accurate: http://release-notes.cloudbees.com/release/Jenkins+OSS+LTS/1.554.1

            Show
            olivergondza Oliver Gondža added a comment - Andrew Erickson : The LTS changelogs are not reliable in this regard. You are right to say that fix went into 1.554.1. See cloudbees changlog that tends to be accurate: http://release-notes.cloudbees.com/release/Jenkins+OSS+LTS/1.554.1
            olivergondza Oliver Gondža made changes -
            Labels 1.554.1-fixed lts-candidate remoting 1.554.1-fixed remoting
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 152578 ] JNJira + In-Review [ 194269 ]

              People

              Assignee:
              kohsuke Kohsuke Kawaguchi
              Reporter:
              rprots Roman Prots'
              Votes:
              8 Vote for this issue
              Watchers:
              24 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: