If you encounter similar problems on recent versions of Jenkins and up to date slave.jar on agents, please file a new bug instead of reopening this one. It's too old to be useful. Provide as much information about your configuration as possible.

I'm now running Jenkins ver. 1.435

My mileage varied too... I got the stream corruption error eventually... so that version isn't as safe as I was professing.

I did notice issue JENKINS-8856, which got stream corruption errors when the master JVM was 64-bit and the slave JVM was 32-bit. I have the same setup but reverse (the default JVM for the Master on Solaris is 32-bit, which the default JVM on the Linux slave is 64-bit).

So, I've updated my Jenkins Master (running in a Tomcat instance) to use a 64-bit JVM. I will see if this makes any difference.

UPDATE: Once I've ensured that both JVMs are both 64-bit, my builds have been a lot more reliable. I've even moved back onto later releases of the JVMs, as well as upgrading to the most recent version of Jenkins (1.436). And I'm not getting the Stream Corruption errors I was getting. For now, I'm cautiously optimistic that the root of my problem was the master process running within a 32-bit JVM, with the slave running within a 64-bit JVM.

We had also a couple of situations ending up in IOExceptions with Unexpected termination of the channel. In our case we used Linux slaves, which run out of memory. The oom_killer killed the java process running the jenkins agent. This ended up in Unexpacted termination of the channel. Perhaps others had the same problem...

The problem've still randomly occurred.

FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
Page generated: Jan 23, 2012 2:35:42 PMJenkins ver. 1.447

I'm now running Jenkins ver. 1.448

Hi Andrew, given this issue is assigned to you, do you have thoughts on how to evaluate it? Any specific input users can provide to help?

My slave machine was working just fine and suddenly I started to see this problem when trying to launch the service from slave. Any thoughts anyone????

I get this message

java.io.EOFException: unexpected stream termination
at hudson.remoting.Channel.<init>(Channel.java:408)
at hudson.remoting.Channel.<init>(Channel.java:366)
at hudson.remoting.Channel.<init>(Channel.java:327)
at hudson.remoting.Channel.<init>(Channel.java:323)
at hudson.remoting.Channel.<init>(Channel.java:311)
at hudson.remoting.Engine.run(Channel.java:238)

@Atiq: doesn't look like the usual crashes I get, e.g. "Unexpected termination of the channel"
So, that's good – there's probably a fix for your problem.

I'm now running

Page generated: Mar 26, 2012 8:17:27 AM
Jenkins ver. 1.456

and still having these exceptions.

I was having this problem. The node had this:

• Debian Lenny i386
• Jenkins ver 1.456
• Java is sun-java6-jdk 6-26-9lenny1 (see http://packages.qa.debian.org/s/sun-java6.html for more version info)

I was able to resolve the issue by removing the package java-gcj-compat-headless.
This may not work for everyone but perhaps it is a clue to what is going wrong.

Hi Vince,

Um, you're saying you opened up a jar file and deleted the package?
On the Jenkins server or on a node? (Or both?)
What gave you the idea that this would help?

Thanks,
-John

I had this problem today, and it turned out to be because swap space was off, and Java was trying to allocate a very large amount of virtual memory.

HTH

Hi,

I´m not sure if this is the right place to write about our observations about channel termination.
Several times a day Jenkins loses some slaves. Currently we are running Jenkins 1.454, but we saw the problem also with 1.457. The master and almost all slaves are running on Linux machines.

I suspect the problem has something to do with garbage collection.
When the problem occurs, Jenkins occupies all 24 hyper-threads of CPU almost 100%.
Here the heap usage (from space 99%):

 
Heap
 PSYoungGen      total 7202048K, used 23148K [0x0000000600000000, 0x0000000800000000, 0x0000000800000000)
  eden space 7181888K, 0% used [0x0000000600000000,0x00000006002ecb30,0x00000007b6590000)
  from space 20160K, 99% used [0x00000007b6590000,0x00000007b793e4f8,0x00000007b7940000)
  to   space 26624K, 0% used [0x00000007fe600000,0x00000007fe600000,0x0000000800000000)
 PSOldGen        total 8388608K, used 2438211K [0x0000000200000000, 0x0000000400000000, 0x0000000600000000)
  object space 8388608K, 29% used [0x0000000200000000,0x0000000294d10d10,0x0000000400000000)
 PSPermGen       total 1048576K, used 85742K [0x00000001c0000000, 0x0000000200000000, 0x0000000200000000)
  object space 1048576K, 8% used [0x00000001c0000000,0x00000001c53bb888,0x0000000200000000)

Excerpt from Jenkins log (master)

 
Apr 24, 2012 11:14:16 AM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel ullteb15
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
Apr 24, 2012 11:14:16 AM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel ullteb16
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
Apr 24, 2012 11:14:26 AM hudson.remoting.Request$2 run
SEVERE: Failed to send back a reply
java.io.IOException: Broken pipe
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(Unknown Source)
	at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
	at java.io.BufferedOutputStream.flush(Unknown Source)
	at java.io.ObjectOutputStream$BlockDataOutputStream.flush(Unknown Source)
	at java.io.ObjectOutputStream.flush(Unknown Source)
	at hudson.remoting.Channel.send(Channel.java:505)
	at hudson.remoting.Request$2.run(Request.java:301)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Apr 24, 2012 11:14:26 AM hudson.remoting.Channel$ReaderThread run
SEVERE: I/O error in channel ulcppit01
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
Apr 24, 2012 11:14:27 AM hudson.remoting.Request$2 run
SEVERE: Failed to send back a reply
java.io.IOException: Broken pipe
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(Unknown Source)
	at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
	at java.io.BufferedOutputStream.flush(Unknown Source)
	at java.io.ObjectOutputStream$BlockDataOutputStream.flush(Unknown Source)
	at java.io.ObjectOutputStream.flush(Unknown Source)
	at hudson.remoting.Channel.send(Channel.java:505)
	at hudson.remoting.Request$2.run(Request.java:301)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Apr 24, 2012 11:14:26 AM hudson.remoting.Request$2 run
SEVERE: Failed to send back a reply

And here log from slave ullteb15

 
Ping failed. Terminating
ERROR: Connection terminated
ha:AAAAWB+LCAAAAAAAAABb85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1133)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)
ERROR: Process terminated with exit code 255
ha:AAAAWB+LCAAAAAAAAABb85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=

Log from slave ullteb28

Apr 24, 2012 11:13:56 AM hudson.slaves.ChannelPinger$1 onDead
INFO: Ping failed. Terminating the channel.
java.util.concurrent.TimeoutException: Ping started on 1335258596049 hasn't completed at 1335258836049
	at hudson.remoting.PingThread.ping(PingThread.java:114)
	at hudson.remoting.PingThread.run(PingThread.java:81)
Caused by: java.util.concurrent.TimeoutException
	at hudson.remoting.Request$1.get(Request.java:249)
	at hudson.remoting.Request$1.get(Request.java:184)
	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59)
	at hudson.remoting.PingThread.ping(PingThread.java:107)
	... 1 more
Apr 24, 2012 11:13:56 AM hudson.slaves.ChannelPinger$1 onDead
INFO: Ping failed. Terminating the channel.
java.util.concurrent.TimeoutException: Ping started on 1335258596049 hasn't completed at 1335258836053
	at hudson.remoting.PingThread.ping(PingThread.java:114)
	at hudson.remoting.PingThread.run(PingThread.java:81)
Caused by: java.util.concurrent.TimeoutException
	at hudson.remoting.Request$1.get(Request.java:249)
	at hudson.remoting.Request$1.get(Request.java:184)
	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59)
	at hudson.remoting.PingThread.ping(PingThread.java:107)
	... 1 more
Connection terminated
channel stopped

Hi,

I did some investigation on this problem over last two months. My case was that this was happening on different linux flavors (but not on Windows build nodes). This started to happen after intensity and number of build jobs started to increase and was happening only for Scala projects. Build nodes are not unique, environment is mixed (some nodes are local, some nodes are on AWS, using two different types of VPN connection).

So after digging more and more, I figured out actually that my kernel wasn't tuned correctly. I increased number of file descriptors and maximum number of processes (as by guys before me it was never tuned) and jobs were doing much more faster, but this started to cause more and more issues.

So, it turned up that this was issue with RedHat based distribution and SSH - no matter I set /etc/security/limits.conf not all values (including this one with nproc setting) were set after SSH connection initiated from Jenkins master server to client(s). I tested it easily by making simple bash script which will connect from master using SSH and same user we use for builds to build nodes and issue ulimit -u and the value was default (1024).

Quick fix was to propagate change to all linux nodes to .bashrc and add explicitly 'ulimit -u value' there after what this stopped to occur and didn't happen anymore for a ten days now (and was happening multiple times a day, even hour). Also, if you want to tune it more elegantly, you should add 'session required pam_lmits.so' to /etc/pam.d/login what will force every new SSH connection to use defined limits from /etc/security/limits.conf.

Maybe this is not real fix for this issue, but this resolved my problem (and now, after thinking about structure of my jobs, intensity, requirements, etc. and basic values of limits, it was quite logical that this was happening to me) and maybe this will help as well someone else.

For us, the cause of this error was our build slaves (VMs) running out of memory and self-rebooting.

We had a similar issue to this (but with Linux slaves) that was resolved by switching to Oracle's JDK from Open JDK.

Even we face these issues frequently when performing svn operations from Jenkins.
Not sure what went wrong. Can anyone Explain what may be the reason ?

Error log:
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Request.call(Request.java:174)
at hudson.remoting.Channel.call(Channel.java:672)
at hudson.FilePath.act(FilePath.java:854)
at hudson.FilePath.act(FilePath.java:838)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:743)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:685)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1342)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:683)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:588)
at hudson.model.Run.execute(Run.java:1567)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:237)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Request.abort(Request.java:299)
at hudson.remoting.Channel.terminate(Channel.java:732)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2570)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1314)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368)
at hudson.remoting.Command.readFrom(Command.java:92)
at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

Hi,

our team also started to get this kind of error after upgrading subversion plugin from 1.34 to 1.45, jenkins version 1.505 java jvm: hotspot, jre: 1.6.0_22 And this is strange that this started to show up after subversion plugin update. This is not showing on testing jenkins where subversion plugin was 1.40 and was updated to 1.45 and everything works fine there.

FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 	at hudson.remoting.Request.call(Request.java:174)
 	at hudson.remoting.Channel.call(Channel.java:672)
 	at hudson.FilePath.act(FilePath.java:893)
 	at hudson.FilePath.act(FilePath.java:877)
 	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:843)
 	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:781)
 	at hudson.model.AbstractProject.checkout(AbstractProject.java:1369)
 	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:676)
 	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:581)
 	at hudson.model.Run.execute(Run.java:1575)
 	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
 	at hudson.model.ResourceController.execute(ResourceController.java:88)
 	at hudson.model.Executor.run(Executor.java:241)
 Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 	at hudson.remoting.Request.abort(Request.java:299)
 	at hudson.remoting.Channel.terminate(Channel.java:732)
 	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
 Caused by: java.io.IOException: Unexpected termination of the channel
 	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
 Caused by: java.io.EOFException
 	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553)
 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
 	at hudson.remoting.Command.readFrom(Command.java:92)
 	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
 	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

May 10, 2013 1:09:25 PM hudson.remoting.SynchronousCommandTransport$ReaderThread run
SEVERE: I/O error in channel slave_server
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2553)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
	at hudson.remoting.Command.readFrom(Command.java:92)
	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

I had this problem when I witch my slave node with a low end system (an embedded system, 1GHz CPU, 1G RAM, Fedora OS). All things were well before I replaced it. So I'm sure this is caused by the java memory usage.

After some searching from internet, I added one option to the salve node advanced "JVM Options" with "-Xmx256m". And this problem is gone. I don't think it is applied for others, it is depended on user's case, some one has this error because java can't get enough memory, and other may has this error because java exhaust system memory.

Sorry, my comments above(deleted) is wrong. This bug is similar with https://issues.jenkins-ci.org/browse/JENKINS-13341 , switching the slave to use Oracle's JDK (1.8.0, early access) indeed fixed it.

We are also getting this rather disconcertingly often on linux slaves. Is there a sar command that I can use to demonstrate a system resource scarcity during the execution of these builds? Because our builds are not even starting and all the SAR data that I have looked at tells me this is NOT a resource issue on the slaves. BUT I am not a sar master and perhaps I am looking at the wrong things.

Started by upstream project "dp_10.0_begin_nightly" build number 107
originally caused by:
Started by timer
Building remotely on robusta in workspace /home/hudson/ci/jenkins/slaves/robusta/workspace/dp_10.0_marker_nightly
hudson.util.IOException2: remote file operation failed: /home/hudson/ci/jenkins/slaves/robusta/workspace/dp_10.0_marker_nightly at hudson.remoting.Channel@789e3ac5:robusta
at hudson.FilePath.act(FilePath.java:900)
at hudson.FilePath.act(FilePath.java:877)
at hudson.FilePath.mkdirs(FilePath.java:1017)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1365)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:674)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:579)
at hudson.model.Run.execute(Run.java:1575)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:237)
Caused by: hudson.remoting.ChannelClosedException: channel is already closed
at hudson.remoting.Channel.send(Channel.java:494)
at hudson.remoting.Request.call(Request.java:129)
at hudson.remoting.Channel.call(Channel.java:672)
at hudson.FilePath.act(FilePath.java:893)
... 10 more
Caused by: java.io.IOException
at hudson.remoting.Channel.close(Channel.java:910)
at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110)
at hudson.remoting.PingThread.ping(PingThread.java:120)
at hudson.remoting.PingThread.run(PingThread.java:81)
Caused by: java.util.concurrent.TimeoutException: Ping started on 1376069782625 hasn't completed at 1376070022626
... 2 more
Build step 'Groovy Postbuild' marked build as failure

This one almost made it:

[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Processing JUnit
[xUnit] [INFO] - [JUnit] - 114 test report file(s) were found with the pattern 'src/system-tests/smoke/test/reports*/*.xml' relative to '/home/jenkins/jobs/workspace/bla' for the testing framework 'JUnit'.
ERROR: Publisher org.jenkinsci.plugins.xunit.XUnitPublisher aborted due to exception
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
	at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
	at hudson.remoting.Request.call(Request.java:174)
	at hudson.remoting.Channel.call(Channel.java:714)
	at hudson.FilePath.act(FilePath.java:898)
	at hudson.FilePath.act(FilePath.java:882)
	at org.jenkinsci.plugins.xunit.XUnitPublisher.performTests(XUnitPublisher.java:184)
	at org.jenkinsci.plugins.xunit.XUnitPublisher.performXUnit(XUnitPublisher.java:119)
	at org.jenkinsci.plugins.xunit.XUnitPublisher.perform(XUnitPublisher.java:93)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:776)
	at hudson.model.Build$BuildExecution.post2(Build.java:183)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726)
	at hudson.model.Run.execute(Run.java:1628)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:247)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.abort(Request.java:299)
	at hudson.remoting.Channel.terminate(Channel.java:774)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2595)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1988)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1912)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1703)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1341)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1988)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1912)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1988)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1912)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
	at hudson.remoting.Command.readFrom(Command.java:92)
	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:71)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
ERROR: Publisher hudson.tasks.Mailer aborted due to exception
hudson.remoting.ChannelClosedException: channel is already closed
	at hudson.remoting.Channel.send(Channel.java:516)
	at hudson.remoting.Request.call(Request.java:129)
	at hudson.remoting.Channel.call(Channel.java:714)
	at hudson.EnvVars.getRemote(EnvVars.java:212)
	at hudson.model.Computer.getEnvironment(Computer.java:908)
	at jenkins.model.CoreEnvironmentContributor.buildEnvironmentFor(CoreEnvironmentContributor.java:40)
	at hudson.model.Job.getEnvironment(Job.java:374)
	at hudson.model.AbstractProject.getEnvironment(AbstractProject.java:353)
	at hudson.model.Run.getEnvironment(Run.java:2077)
	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:933)
	at hudson.tasks.Mailer.perform(Mailer.java:114)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:776)
	at hudson.model.Build$BuildExecution.post2(Build.java:183)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726)
	at hudson.model.Run.execute(Run.java:1628)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:247)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2595)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1988)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1912)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1703)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1341)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1988)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1912)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1988)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1912)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
	at hudson.remoting.Command.readFrom(Command.java:92)
	at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:71)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

Hi, for my issue investigation pointed to svnkit, in more general gnome-keyring glib is crashing VM. I found that this issue is also connected to subversion checkout lock.

Mainly problem is when couple of builds are running on slave and randomly some build checkout's from subversion( slave's VM crashes) and for other builds in the log it seams that connections was dropped

Solution is to add java vm variable:
-Dsvnkit.library.gnome-keyring.enabled=false

Info on svn lock during checkout with thread dump:
https://issues.jenkins-ci.org/browse/JENKINS-16427

P.S.
Tested on couple of Jenkins instances and linux slaves. Since introducing vm variable jenkins works stable for almost week(before, one of Jenkins instance slaves crashed everyday).

In my case, this exceptions seems to be the result of an overloaded machine. I don't think this is something you "fix" so much as I'd like Jenkins to let me increase the timeout value so it doesn't close the channel for such overloaded machines.

I did the following according to Chris Z's post:

• added -Dsvnkit.library.gnome-keyring.enabled=false to the slave launch options
• set "password-stores = kwallet keychain windows-cryptoapi" in the .subversion/config file on the master

Still ran into this...

We are seeing this constantly on our build slaves during gcc and ARM compilation:

FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
at hudson.remoting.Request.call(Request.java:174)
at hudson.remoting.Channel.call(Channel.java:722)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:167)
at com.sun.proxy.$Proxy47.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:925)
at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:585)
at hudson.model.Run.execute(Run.java:1676)
at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:231)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.Request.abort(Request.java:299)
at hudson.remoting.Channel.terminate(Channel.java:782)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at hudson.remoting.Command.readFrom(Command.java:92)
at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:71)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

I've completely turned off the Response Time check, bumped the heap size to 512MB (monitoring indicates that this is ample), ensured that the slave is using Oracle JRE (1.6.0_45-b06).

Master is Ubuntu 10.04 (fully updated) 8 core, 16GB RAM physical host with Jenkins 1.555.
Slave is Ubuntu 10.04 (fully updated) 4 core, 1GB RAM living on Suse KVM host. Monitoring indicates that CPU consumption and load average are definitely high (compilation, etc).

This is a really ugly issue. What is the status of development?

You know, I could be wrong, but it seems I get this error message less after having switched all my machines from WIFI to wired Ethernet.

This is killing me, I get this 6 times a day but only on my windows slaves. The amount of resources I give to them makes no difference to the chance of failure (they are all quad core with 6Gb of RAM now)

I've been able to "sort of" get around this by only sending IO and CPU heavy stuff to dedicated, powerful systems, reducing the number of executors on each slave to 1, reducing the number of threads to half the number of cores.

Totally lame workaround, but "done at all" is better than the stupid stack dump.

Please make sure to specify Jenkins versions when adding "me too" comments.

We have seen these issues in an unreliable network but one tweak that has helped a lot is to remove the monitoring plugin. That plugin has the unfortunate property of severing connections to slaves if they don't respond to a single ping. The connection will be severed regardless of the slave running jobs at that time.

@ritzmann: Which monitoring plugin?

Which monitoring plugin?

https://wiki.jenkins-ci.org/display/JENKINS/Monitoring

Same issue her with 1.567:

NOTE: recipe attr-native-2.4.46-r4: task do_configure: Succeeded
FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
at hudson.remoting.Request.call(Request.java:174)
at hudson.remoting.Channel.call(Channel.java:739)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:168)
at com.sun.proxy.$Proxy70.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:951)
at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:159)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at hudson.model.Run.execute(Run.java:1710)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:234)
Caused by: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.abort(Request.java:299)
at hudson.remoting.Channel.terminate(Channel.java:802)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:951)
at hudson.remoting.Channel$2.handle(Channel.java:475)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: hudson.remoting.Channel$OrderlyShutdown
... 3 more
Caused by: Command close created at
at hudson.remoting.Command.<init>(Command.java:56)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:945)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:943)
at hudson.remoting.Channel.close(Channel.java:1026)
at hudson.remoting.Channel.close(Channel.java:1009)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:950)
... 2 more

same here on 1.565.1, slave and master are both in the same LAN on Amazon EC2. Quite a frustrating bug.

I have seen the same with v1.583, I haven't found a solution yet but did anyone ever checked the sshd syslog?

I observed that each time a job fails due to "java.io.IOException: Unexpected termination of the channel" there is a syslog entry for some changes in the underlying ssh connection:
Oct 8 01:30:14 servername sshd[pid]: Generating new 1024 bit RSA key.
Oct 8 01:30:14 servername sshd[pid]: RSA key generation complete.

can anyone check if you see the same?

slaves OS: Scientific Linux release 6.5 (Carbon) and CentOS release 6.5 (Final)

KeyRegenerationInterval
             In protocol version 1, the ephemeral server key is automatically regenerated after this
             many seconds (if it has been used).  The purpose of regeneration is to prevent decrypting
             captured sessions by later breaking into the machine and stealing the keys.  The key is
             never stored anywhere.  If the value is 0, the key is never regenerated.  The default is
             3600 (seconds).

The default value for this setting will generate an ephemerial 1024bit RSA key for the sshd process every 3600 sec. Try setting it to zero?

I suggest doing the opposite. Set the value to a small number, like 10 seconds, to make the issue easier to reproduce and investigate a fix.

@Ian: that's fair enough but I'd expect for ssh key regeneration to be done in the background without breaking the connection.
@cowwoc: I'll try that but I'm not yet sure where to change the setting, should it be on the master node of slave? I'll do a bit of reading on how ssh does this

@Calin Marina : i am using Jenkins LTS versions and i don't have this issue ... to much . Maybe consider switching to LTS. Much more stable in a production enviroment

unfortunately I only have full access to the slaves not on the master server... master is administered by others and I can only lobby for changes for now

that is very bad...especially if you need more access to configure. I am the build manager for my team so it is simpler for all devs because i am very fast to approach

https://issues.jenkins-ci.org/browse/JENKINS-14332 helped for me.

Sometimes I have this problem when running Jenkins 1.637 (Tomcat 8.0.28 / Windows 10)

[WARNING] Couldn't clean up oid=9 from null
hudson.remoting.ChannelClosedException: channel is already closed
	at hudson.remoting.Channel.send(Channel.java:575)
	at hudson.remoting.RemoteInvocationHandler$PhantomReferenceImpl.cleanup(RemoteInvocationHandler.java:360)
	at hudson.remoting.RemoteInvocationHandler$PhantomReferenceImpl.access$700(RemoteInvocationHandler.java:319)
	at hudson.remoting.RemoteInvocationHandler$Unexporter.run(RemoteInvocationHandler.java:420)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:110)
	at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.remoting.Channel$OrderlyShutdown
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1077)
	at hudson.remoting.Channel$1.handle(Channel.java:498)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command close created at
	at hudson.remoting.Command.<init>(Command.java:56)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1071)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1069)
	at hudson.remoting.Channel.close(Channel.java:1153)
	at hudson.remoting.Channel.close(Channel.java:1135)
	at hudson.maven.ProcessCache$MavenProcess.discard(ProcessCache.java:150)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:861)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
	at hudson.model.Run.execute(Run.java:1741)
	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Finished: SUCCESS

In Jenkins ver. 1.625 this problem shows up as:

Slave went offline during the build
ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@18bfcae9[name=Arnold]
	at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
	at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:628)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
	at sun.nio.ch.SocketDispatcher.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(Unknown Source)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
	at sun.nio.ch.IOUtil.read(Unknown Source)
	at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
	at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:136)
	at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:306)
	at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
	... 6 more

Hi John,
I had the same problem one year ago with an older version.
Try to launch the slave with a batch instead of a service and redirect the standard output to a file to check what happens.
Honestly I solved disabling the service and leaving the slave running with a scheduled batch.
Graz

Any resolution for connection timeout issues ?

Ubuntu 14.04 server 64-bit
oracle-java7: 1.7.0_80
Jenkins: 1.651.1 LTS

The build sometimes randomly failed with this kind of error.

This time happened in the post-build actions:
FATAL: channel is already closed
hudson.remoting.ChannelClosedException: channel is already closed
at hudson.remoting.Channel.send(Channel.java:578)
at hudson.remoting.Request.call(Request.java:130)
at hudson.remoting.Channel.call(Channel.java:780)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:953)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:540)
at hudson.model.Run.execute(Run.java:1738)
at hudson.matrix.MatrixBuild.run(MatrixBuild.java:313)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

I got similar issues as below

FATAL: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1467229478054 hasn't completed at 1467229718378
hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1467229478054 hasn't completed at 1467229718378
	at hudson.remoting.Request.abort(Request.java:295)
	at hudson.remoting.Channel.terminate(Channel.java:814)
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1029)
	at hudson.remoting.Channel$2.handle(Channel.java:483)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
	at ......remote call to 10.79.17.177(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356)
	at hudson.remoting.Request.call(Request.java:171)
	at hudson.remoting.Channel.call(Channel.java:751)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:173)
	at $Proxy36.join(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:979)
	at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770)
	at hudson.model.Build$BuildExecution.build(Build.java:199)
	at hudson.model.Build$BuildExecution.doRun(Build.java:160)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533)
	at hudson.model.Run.execute(Run.java:1745)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:89)
	at hudson.model.Executor.run(Executor.java:240)
Caused by: hudson.remoting.Channel$OrderlyShutdown: java.util.concurrent.TimeoutException: Ping started on 1467229478054 hasn't completed at 1467229718378
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1029)
	at hudson.remoting.Channel$2.handle(Channel.java:483)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command close created at
	at hudson.remoting.Command.<init>(Command.java:56)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1023)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1021)
	at hudson.remoting.Channel.close(Channel.java:1104)
	at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110)
	at hudson.remoting.PingThread.ping(PingThread.java:125)
	at hudson.remoting.PingThread.run(PingThread.java:86)
Caused by: java.util.concurrent.TimeoutException: Ping started on 1467229478054 hasn't completed at 1467229718378
	... 2 more    

My environment:
jenkins master runs on CentOS release 6.4 (Final). jenkins version is : 1.580.3
jenkins slave in which am facing issue is a solaris vm. This started recently and I could'nt think of any environment changes happened. When, this happened initially, I thought that swap space is not enough and I have increased the swap space of the slave. Any suggestions is much appreciated. When I hit this issue, I just reboot the slave and then one or two build works fine. Have not seen this problem after three builds, but not sure, when I will hit this.

JENKINS-1948?

Remoting logic has improved gradually in this area (especially on diagnostics fronts), but likely the issue still exists. Not sure if there is a way to fix such generic issue

xiazhang

also encountered the same problems

When commenting on a six years old issue, you need to mention what version of Jenkins you're running. If the problem was trivial to reproduce and fix, this wouldn't still be open after six years.

That said, no need to amend your comment: Based on line numbers, this is remoting 2.60 or older, a lot has changed since then, so you're on a Jenkins version nobody cares about anymore. Time to upgrade.

Hi all. As a Remoting maintainer, I am going to close it. In Remoting 3.x we significantly improved the diagnosability and stability of the channel. In the new versions you would need to...

1. Use JNLP4
2. Use Remoting 3.14+
3. Use -workDir mode with agent-side log collection
4. Use Jenkins 2.89.3+

If such generic issue happens on such configuration, let's follow-up in new tickets so that we can diagnose them on the case-by-case basis.

Closing to discourage new reports in this ticket