Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17757

IllegalStateException: Timer already cancelled from NodesCollector.scheduleCollectNow

XMLWordPrintable

      A user reported that his (SSH) slaves repeatedly failed to launch with the following error:

      ...
Copied classworlds.jar 
Evacuated stdout 
ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins. 
java.lang.IllegalStateException: Timer already cancelled. 
at java.util.Timer.sched(Timer.java:397) 
at java.util.Timer.schedule(Timer.java:193) 
at net.bull.javamelody.NodesCollector.scheduleCollectNow(NodesCollector.java:110) 
at org.jvnet.hudson.plugins.monitoring.NodesListener.onOnline(NodesListener.java:51) 
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:472) 
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:340) 
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:678) 
at hudson.plugins.sshslaves.SSHLauncher.launch(SSHLauncher.java:472) 
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:223) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
at java.lang.Thread.run(Thread.java:722) 
[…] [SSH] Connection closed. 
ERROR: Connection terminated 
java.io.IOException: Unexpected termination of the channel 
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50) 
Caused by: ...

      After disabling the Monitoring plugin the problem went away.

      Not sure what the root cause is, but catching IllegalStateException from scheduleCollectNow and reporting it gracefully with diagnostics seems like a good idea.

      Probably SlaveComputer.setChannel should also be trapping exceptions from listeners it calls. hudson.Util may need a convenience method to call a listener method (as a Runnable? for future lambdas) catching any RuntimeException or LinkageError and politely reporting the error so the caller can continue (maybe even blacklisting the listener for future calls so you do not fill up your log, listing the plugin name and version owning the listener class, etc.).

          [JENKINS-17757] IllegalStateException: Timer already cancelled from NodesCollector.scheduleCollectNow

          Jesse Glick created issue -
          Jesse Glick made changes -
          Description Original: A user reported that his (SSH) slaves repeatedly failed to launch with the following error:

          {code:none}
          ...
          Copied classworlds.jar
          Evacuated stdout
          ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
          java.lang.IllegalStateException: Timer already cancelled.
          at java.util.Timer.sched(Timer.java:397)
          at java.util.Timer.schedule(Timer.java:193)
          at net.bull.javamelody.NodesCollector.scheduleCollectNow(NodesCollector.java:110)
          at org.jvnet.hudson.plugins.monitoring.NodesListener.onOnline(NodesListener.java:51)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:472)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:340)
          at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:678)
          at hudson.plugins.sshslaves.SSHLauncher.launch(SSHLauncher.java:472)
          at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:223)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
          at java.util.concurrent.FutureTask.run(FutureTask.java:166)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
          at java.lang.Thread.run(Thread.java:722)
          [04/17/13 22:18:02] [SSH] Connection closed.
          ERROR: Connection terminated
          java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
          Caused by: ...
          {code}

          After disabling the Monitoring plugin the problem went away.

          Not sure what the root cause is, but catching {{IllegalStateException}} from {{scheduleCollectNow}} and reporting it gracefully with diagnostics seems like a good idea.

          Probably {{SlaveComputer.setChannel}} should also be trapping exceptions from listeners it calls. {{hudson.Util}} may need a convenience method to call a listener method (as a {{Runnable}}? for future lambdas) catching any {{RuntimeException}} or {{LinkageError}} and politely reporting the error so the caller can continue (maybe even blacklisting the listener for future calls so you do not fill up your log, listing the plugin name and version owning the listener class, etc.).           		New: A user reported that his (SSH) slaves repeatedly failed to launch with the following error:

          {code:none}
          ...
          Copied classworlds.jar
          Evacuated stdout
          ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
          java.lang.IllegalStateException: Timer already cancelled.
          at java.util.Timer.sched(Timer.java:397)
          at java.util.Timer.schedule(Timer.java:193)
          at net.bull.javamelody.NodesCollector.scheduleCollectNow(NodesCollector.java:110)
          at org.jvnet.hudson.plugins.monitoring.NodesListener.onOnline(NodesListener.java:51)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:472)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:340)
          at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:678)
          at hudson.plugins.sshslaves.SSHLauncher.launch(SSHLauncher.java:472)
          at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:223)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
          at java.util.concurrent.FutureTask.run(FutureTask.java:166)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
          at java.lang.Thread.run(Thread.java:722)
          […] [SSH] Connection closed.
          ERROR: Connection terminated
          java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
          Caused by: ...
          {code}

          After disabling the Monitoring plugin the problem went away.

          Not sure what the root cause is, but catching {{IllegalStateException}} from {{scheduleCollectNow}} and reporting it gracefully with diagnostics seems like a good idea.

          Probably {{SlaveComputer.setChannel}} should also be trapping exceptions from listeners it calls. {{hudson.Util}} may need a convenience method to call a listener method (as a {{Runnable}}? for future lambdas) catching any {{RuntimeException}} or {{LinkageError}} and politely reporting the error so the caller can continue (maybe even blacklisting the listener for future calls so you do not fill up your log, listing the plugin name and version owning the listener class, etc.).
          evernat made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-21224 [ JENKINS-21224 ]
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 148937 ] New: JNJira + In-Review [ 192951 ]

            Unassigned Unassigned
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: