Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51491

Remoting Thread Terminated Randomly

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Critical Critical
    • remoting
    • None
    • Jenkins Server OS: Windows Server 2012(64-bit)
      Jenkins Version: 2.89.4

      After upgrade Jenkins to 2.98.4 LTS, we found that the slaves always disconnect from server randomly, and from the server side, we checked log info with "thread terminated", how can we know why the thread is terminated? I also checked thread dump, but there are only dump for live thread but not for killed thread. The debug info for Server and Slave see below:

      Debug info from Salves:

      May 23, 2018 10:33:01 AM hudson.remoting.UserRequest perform
      WARNING: LinkageError while performing UserRequest:UserRPCRequest(3,join)
      java.lang.LinkageError: Failed to load hudson.util.ProcessTree
      at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:387)
      at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:295)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at hudson.Proc$LocalProc.destroy(Proc.java:384)
      at hudson.Proc$LocalProc.join(Proc.java:357)
      at hudson.Launcher$RemoteLaunchCallable$1.join(Launcher.java:1304)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922)
      at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896)
      at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
      at hudson.remoting.UserRequest.perform(UserRequest.java:207)
      at hudson.remoting.UserRequest.perform(UserRequest.java:53)
      at hudson.remoting.Request$2.run(Request.java:358)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at hudson.remoting.Engine$1$1.run(Engine.java:98)
      at java.lang.Thread.run(Unknown Source)
      Caused by: java.lang.NoClassDefFoundError: hudson/util/ProcessTreeRemoting$IProcessTree
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(Unknown Source)
      at java.lang.ClassLoader.defineClass(Unknown Source)
      at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383)
      ... 22 more
      Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTreeRemoting$IProcessTree
      at java.net.URLClassLoader.findClass(Unknown Source)
      at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:159)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      ... 26 more

       

      Debug Info from Server:
      May 23, 2018 10:32:55 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
      INFO: Accepted JNLP4-connect connection #101,988 from /10.224.105.34:28333
      May 23, 2018 10:32:57 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
      WARNING: IOHub#1: Worker[channel:java.nio.channels.SocketChannel[connected local=/10.224.105.17:53233 remote=avcndaevpcisrv/10.224.105.34:1655]] / Computer.threadPoolForRemoting 5173 for FTVP_CI_Slv_SSD_View terminated
      java.nio.channels.ClosedChannelException
      at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
      at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:179)
      at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:721)
      at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)

            jthompson Jeff Thompson
            ljie Jay Jie
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: