I'm not yet sure whether that's related to JENKINS-18781, and JENKINS-22758 seems to be solved already. I'm experiencing strange issues with the connection between Jenkins master and its slaves (connected using the swarm plugin) during startup. All the >25 slaves repeatable disconnect and automatically reconnect for quite some time:
- grep -c 'WARNING: Channel reader thread: .* terminated' ~log/jenkins/jenkins.log
2882
The data for the slaves at $JENKINS_URL/computer/ is missing for quite some time, but finally they persistent and then they seem to work fine. Was there some timeout decreased recently which might explain that issue?
Quoting Jenkins master's log:
Aug 12, 2014 12:43:08 AM jenkins.slaves.JnlpSlaveAgentProtocol$Handler$1 onClosed
WARNING: Channel reader thread: docker1 for + docker1 terminated
java.nio.channels.AsynchronousCloseException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:205)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:412)
at hudson.remoting.SocketChannelStream$1.read(SocketChannelStream.java:33)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.io.InputStream.read(InputStream.java:101)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:81)
at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2293)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2586)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1318)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at hudson.remoting.Command.readFrom(Command.java:92)
at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:70)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
On the client that's in the logs:
Aug 12, 2014 12:43:06 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run
SEVERE: I/O error in channel channel
java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2598)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1318)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at hudson.remoting.Command.readFrom(Command.java:92)
at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:71)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
Do you have the whole master log? AsynchronousCloseException indicates that another thread in the master closed the connection (and EOFException on the slave is consistent with that). And hopefully that another thread (perhaps ping thread, or perhaps NIO selector thread) should have left the record of that around the same time.
If you scroll up and down around that AsynchronousCloseException, maybe you'll find it.