Search

Here is more information about the sequence of events.
1)The Channel Pinger fails and it immediately terminates the connection channel to the target slave node
2)The thread that was running on the target slave node is aborted
3)The system attempts a reconnect to the target slave node

Here is the shortened version of the log statements
====================================================
Jan 24, 2012 3:35:28 PM hudson.slaves.ChannelPinger$1 onDead
INFO: Ping failed. Terminating the channel.

Exception in thread "Monitoring slave-node1 for Free Swap Space" hudson.remoting.RequestAbortedException: hud
son.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown

Jan 24, 2012 3:36:43 PM hudson.slaves.SlaveComputer tryReconnect
INFO: Attempting to reconnect slave-node1

I am wondering if the Channel Pinger upon encountering a network error terminates the communication channel immediately or does it do a few reties before it gives up? If possible a retry algorithm may be a good option since network/routing errors can happen from time to time.

One more piece of information. This issue happened again the other day while the target build slave server was cloning a repository. This means that the build slave server was very busy doing I/O and cpu activities.

Jenkins Log in jenkins.log file
================================================================================================
Feb 15, 2012 5:35:12 PM hudson.slaves.ChannelPinger$1 onDead
INFO: Ping failed. Terminating the channel.
Exception in thread "Monitoring slave-07 for Free Swap Space" hudson.remoting.RequestAbortedException: hud
son.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.call(Request.java:149)
at hudson.remoting.Channel.call(Channel.java:660)
at hudson.node_monitors.SwapSpaceMonitor$1.monitor(SwapSpaceMonitor.java:83)
at hudson.node_monitors.SwapSpaceMonitor$1.monitor(SwapSpaceMonitor.java:81)
at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:
202)
Caused by: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.abort(Request.java:269)
at hudson.remoting.Channel.terminate(Channel.java:711)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:794)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1024)
Caused by: hudson.remoting.Channel$OrderlyShutdown
... 2 more
Caused by: Command close created at
at hudson.remoting.Command.<init>(Command.java:62)
at hudson.remoting.Command.<init>(Command.java:47)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:790)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:790)
at hudson.remoting.Channel.close(Channel.java:835)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:793)
... 1 more
Feb 15, 2012 5:35:12 PM hudson.model.AbstractBuild$AbstractRunner performAllBuildSteps
WARNING: Publisher hudson.tasks.junit.JUnitResultArchiver aborted due to exception
java.lang.NullPointerException
at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:83)
at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:123)
at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:135)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649)
at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:625)
at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:603)
at hudson.model.Build$RunnerImpl.post2(Build.java:161)
at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:572)
at hudson.model.Run.run(Run.java:1386)
at hudson.matrix.MatrixRun.run(MatrixRun.java:137)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
Feb 15, 2012 5:35:53 PM hudson.slaves.SlaveComputer tryReconnect
INFO: Attempting to reconnect slave-07

Build Job failure messages
==================================================================================
FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.call(Request.java:149)
at hudson.remoting.Channel.call(Channel.java:660)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
at $Proxy17.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:850)
at hudson.Launcher$ProcStarter.join(Launcher.java:336)
at hudson.plugins.mercurial.MercurialSCM.clone(MercurialSCM.java:577)
at hudson.plugins.mercurial.MercurialSCM.checkout(MercurialSCM.java:422)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1174)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:523)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:418)
at hudson.model.Run.run(Run.java:1362)
at hudson.matrix.MatrixRun.run(MatrixRun.java:137)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
Caused by: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Request.abort(Request.java:269)
at hudson.remoting.Channel.terminate(Channel.java:711)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:794)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1024)
Caused by: hudson.remoting.Channel$OrderlyShutdown
... 2 more
Caused by: Command close created at
at hudson.remoting.Command.<init>(Command.java:62)
at hudson.remoting.Command.<init>(Command.java:47)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:790)
at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:790)
at hudson.remoting.Channel.close(Channel.java:835)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:793)
... 1 more

Is it reproduced with a recent Jenkins version?

We have the same issue on Windows Server 2008R2, x64, Microsoft Hyper-V (both master and slaves).
JRE 1.7.0, Jenkins 1.509.4 LTS.