Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59885

"Agent went offline" or "Channel alread closed" during the build

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: component
    • Environment:
    • Similar Issues:

      Description

      The main job is started, which is starting a some other (sub)jobs. Mostly at the same sub-job, after the first four jobs has been executed without any problem, a job is started. When that job is started or if this job runs one or three minutes we get the issue "Agent went offline". When it runs like say 4 min. than it runs to the end without any issue! That job runs for 1,5 hour or even more!

      After it went wrong with "Agent went offliner" or with "Channel already closed" we start the main job again and than it works always without a issue!!!!????? WHY?????

      Analyzing this:

      • Windows event viewer (see screenshot), I can see that the Jenkinsslave-C_Jenkins is "normally" stopped. But by whom? The Job is not doing this or at least so far as I can see the Job is not doing this and should not do this. See my screenshot. In this screenshot it's a few seconds later the service is stopped. But I have other circumstances and then it was in the exact same second!
      • Jenkins logging "Jenkins-slave.err". There I see issue's with JNLP connection but on this moment I don't know what to do about it! What do I have to change or were can I find documentation. 

      INFO: Trying protocol: JNLP3-connect
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP2-connect
      Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Oct 21, 2019 7:12:21 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Terminated
      Oct 21, 2019 7:12:31 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
      INFO: Restarting agent via jenkins.slaves.restarter.WinswSlaveRestarter@48eae634
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main createEngine
      INFO: Setting up slave: nodename
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Jenkins agent is running in headless mode.
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among http://<ip adres>/jenkins/
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Handshaking
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP3-connect
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP2-connect
      Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main createEngine
      INFO: Setting up slave: nodename
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Jenkins agent is running in headless mode.
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among http://<ip adress>jenkins/
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Handshaking
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP3-connect
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to <ip adress>:<fixed port adress>
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP2-connect
      Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.
      Oct 21, 2019 10:51:41 PM hudson.plugins.tmpcleaner.TmpCleanTask call
      INFO: end TmpCleanTask
      Oct 22, 2019 4:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
      INFO: end TmpCleanTask
      Oct 22, 2019 10:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
      INFO: end TmpCleanTask

      • The console log Agent went offline at the same time the JNLP is TERMINATED in Jenkins-slave.err file!:

      19:12:10 2019-10-21
      19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
      19:12:10 2019-10-21 19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
      19:12:10 2019-10-21 19:12:10.3153|INFO|Deploy.Program|File upload successfully done: yyy.json
      19:12:21 Agent went offline during the build19:12:21 ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2838043[name=<node_name>]
      19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
      19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:629)19:12:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      19:12:21 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)19:12:21 at java.util.concurrent.FutureTask.run(FutureTask.java:266)19:12:21 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      19:12:21 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      19:12:21 at java.lang.Thread.run(Thread.java:745)
      19:12:21 Caused by: java.io.IOException: Connection timed out19:12:21 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
      19:12:21 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
      19:12:21 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
      19:12:21 at sun.nio.ch.IOUtil.read(IOUtil.java:197)19:12:21 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
      19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:137)
      19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:310)
      19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
      19:12:21 ... 6 more19:12:21 

      • Screenshot configuring the node

       

      I hope that someone can help me out this issue. When go DuckDuckGo than I find out I'm not the only one who have this issue but I can't find a solution. 

      Regards,

      Nico

        Attachments

          Activity

          nkamp01 Nico van de Kamp created issue -
          nkamp01 Nico van de Kamp made changes -
          Field Original Value New Value
          Description The main job is started, which is starting a some other (sub)jobs. Mostly at the same sub-job, after the first four jobs has been executed without any problem, a job is started. When that job is started or if this job runs one or three minutes we get the issue "Agent went offline". When it runs like say 4 min. than it runs to the end without any issue! That job runs for 1,5 hour or even more!

          After it went wrong with "Agent went offliner" or with "Channel already closed" we start the main job again and than it works always without a issue!!!!????? WHY?????

          Analyzing this:
           * *Windows event viewer* (see screenshot), I can see that the Jenkinsslave-C_Jenkins is "normally" stopped. But by whom? The Job is not doing this or at least so far as I can see the Job is not doing this and should not do this. See my screenshot. In this screenshot it's a few seconds later the service is stopped. But I have other circumstances and then it was in the exact same second!
           * *Jenkins logging "Jenkins-slave.err"*. There I see issue's with JNLP connection but on this moment I don't know what to do about uit! What do I have to change or were can I find documentation. 

          INFO: Trying protocol: JNLP3-connect
          Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
          Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connecting to <ip adress>:<fixed port adress>
          Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Trying protocol: JNLP2-connect
          Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connected
          *{color:#FF0000}Oct 21, 2019 7:12:21 PM hudson.remoting.jnlp.Main$CuiListener status{color}*
          *{color:#FF0000}INFO: Terminated{color}*
          Oct 21, 2019 7:12:31 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
          INFO: Restarting agent via jenkins.slaves.restarter.WinswSlaveRestarter@48eae634
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main createEngine
          INFO: Setting up slave: nodename
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener <init>
          INFO: Jenkins agent is running in headless mode.
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Locating server among [http://&lt;ip adres>/jenkins/]
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Handshaking
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connecting to <ip adress>:<fixed port adress>
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Trying protocol: JNLP3-connect
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connecting to <ip adress>:<fixed port adress>
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Trying protocol: JNLP2-connect
          Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connected
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main createEngine
          INFO: Setting up slave: nodename
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener <init>
          INFO: Jenkins agent is running in headless mode.
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Locating server among [http://&lt;ip adress>jenkins/]
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Handshaking
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connecting to <ip adress>:<fixed port adress>
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Trying protocol: JNLP3-connect
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connecting to <ip adress>:<fixed port adress>
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Trying protocol: JNLP2-connect
          Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connected
          Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.
          Oct 21, 2019 10:51:41 PM hudson.plugins.tmpcleaner.TmpCleanTask call
          INFO: end TmpCleanTask
          Oct 22, 2019 4:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
          INFO: end TmpCleanTask
          Oct 22, 2019 10:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
          INFO: end TmpCleanTask
           * *The console log* Agent went offline at the same time the JNLP is TERMINATED in Jenkins-slave.err file!:

          19:12:10 2019-10-21
          19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
          19:12:10 2019-10-21 19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
          19:12:10 2019-10-21 19:12:10.3153|INFO|Deploy.Program|File upload successfully done: yyy.json
          *{color:#FF0000}19:12:21 Agent went offline during the build19:12:21{color}* ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2838043[name=<node_name>]
          19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
          19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:629)19:12:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          19:12:21 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)19:12:21 at java.util.concurrent.FutureTask.run(FutureTask.java:266)19:12:21 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          19:12:21 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          19:12:21 at java.lang.Thread.run(Thread.java:745)
          19:12:21 Caused by: java.io.IOException: Connection timed out19:12:21 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
          19:12:21 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
          19:12:21 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
          19:12:21 at sun.nio.ch.IOUtil.read(IOUtil.java:197)19:12:21 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
          19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:137)
          19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:310)
          19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
          19:12:21 ... 6 more19:12:21 
           * *Screenshot configuring the node*

           

          *I hope that someone can help me out this issue. When go DuckDuckGo than I find out I'm not the only one who have this issue but I can't find a solution.* 

          *Regards,*

          *Nico*
          The main job is started, which is starting a some other (sub)jobs. Mostly at the same sub-job, after the first four jobs has been executed without any problem, a job is started. When that job is started or if this job runs one or three minutes we get the issue "Agent went offline". When it runs like say 4 min. than it runs to the end without any issue! That job runs for 1,5 hour or even more!

          After it went wrong with "Agent went offliner" or with "Channel already closed" we start the main job again and than it works always without a issue!!!!????? WHY?????

          Analyzing this:
           * *Windows event viewer* (see screenshot), I can see that the Jenkinsslave-C_Jenkins is "normally" stopped. But by whom? The Job is not doing this or at least so far as I can see the Job is not doing this and should not do this. See my screenshot. In this screenshot it's a few seconds later the service is stopped. But I have other circumstances and then it was in the exact same second!
           * *Jenkins logging "Jenkins-slave.err"*. There I see issue's with JNLP connection but on this moment I don't know what to do about it! What do I have to change or were can I find documentation. 

          INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           *{color:#ff0000}Oct 21, 2019 7:12:21 PM hudson.remoting.jnlp.Main$CuiListener status{color}*
           *{color:#ff0000}INFO: Terminated{color}*
           Oct 21, 2019 7:12:31 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
           INFO: Restarting agent via jenkins.slaves.restarter.WinswSlaveRestarter@48eae634
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main createEngine
           INFO: Setting up slave: nodename
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener <init>
           INFO: Jenkins agent is running in headless mode.
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Locating server among [http://&lt;ip adres>/jenkins/|http://%3Cip%20adres%3E/jenkins/]
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Handshaking
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main createEngine
           INFO: Setting up slave: nodename
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener <init>
           INFO: Jenkins agent is running in headless mode.
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Locating server among [http://&lt;ip adress>jenkins/|http://%3Cip%20adress%3Ejenkins/]
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Handshaking
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.
           Oct 21, 2019 10:51:41 PM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           Oct 22, 2019 4:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           Oct 22, 2019 10:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           * *The console log* Agent went offline at the same time the JNLP is TERMINATED in Jenkins-slave.err file!:

          19:12:10 2019-10-21
           19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
           19:12:10 2019-10-21 19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
           19:12:10 2019-10-21 19:12:10.3153|INFO|Deploy.Program|File upload successfully done: yyy.json
           *{color:#ff0000}19:12:21 Agent went offline during the build19:12:21{color}* ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2838043[name=<node_name>]
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:629)19:12:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
           19:12:21 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)19:12:21 at java.util.concurrent.FutureTask.run(FutureTask.java:266)19:12:21 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           19:12:21 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           19:12:21 at java.lang.Thread.run(Thread.java:745)
           19:12:21 Caused by: java.io.IOException: Connection timed out19:12:21 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
           19:12:21 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
           19:12:21 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
           19:12:21 at sun.nio.ch.IOUtil.read(IOUtil.java:197)19:12:21 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
           19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:137)
           19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:310)
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
           19:12:21 ... 6 more19:12:21 
           * *Screenshot configuring the node*

           

          *I hope that someone can help me out this issue. When go DuckDuckGo than I find out I'm not the only one who have this issue but I can't find a solution.* 

          *Regards,*

          *Nico*
          nkamp01 Nico van de Kamp made changes -
          Description The main job is started, which is starting a some other (sub)jobs. Mostly at the same sub-job, after the first four jobs has been executed without any problem, a job is started. When that job is started or if this job runs one or three minutes we get the issue "Agent went offline". When it runs like say 4 min. than it runs to the end without any issue! That job runs for 1,5 hour or even more!

          After it went wrong with "Agent went offliner" or with "Channel already closed" we start the main job again and than it works always without a issue!!!!????? WHY?????

          Analyzing this:
           * *Windows event viewer* (see screenshot), I can see that the Jenkinsslave-C_Jenkins is "normally" stopped. But by whom? The Job is not doing this or at least so far as I can see the Job is not doing this and should not do this. See my screenshot. In this screenshot it's a few seconds later the service is stopped. But I have other circumstances and then it was in the exact same second!
           * *Jenkins logging "Jenkins-slave.err"*. There I see issue's with JNLP connection but on this moment I don't know what to do about it! What do I have to change or were can I find documentation. 

          INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           *{color:#ff0000}Oct 21, 2019 7:12:21 PM hudson.remoting.jnlp.Main$CuiListener status{color}*
           *{color:#ff0000}INFO: Terminated{color}*
           Oct 21, 2019 7:12:31 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
           INFO: Restarting agent via jenkins.slaves.restarter.WinswSlaveRestarter@48eae634
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main createEngine
           INFO: Setting up slave: nodename
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener <init>
           INFO: Jenkins agent is running in headless mode.
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Locating server among [http://&lt;ip adres>/jenkins/|http://%3Cip%20adres%3E/jenkins/]
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Handshaking
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main createEngine
           INFO: Setting up slave: nodename
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener <init>
           INFO: Jenkins agent is running in headless mode.
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Locating server among [http://&lt;ip adress>jenkins/|http://%3Cip%20adress%3Ejenkins/]
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Handshaking
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.
           Oct 21, 2019 10:51:41 PM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           Oct 22, 2019 4:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           Oct 22, 2019 10:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           * *The console log* Agent went offline at the same time the JNLP is TERMINATED in Jenkins-slave.err file!:

          19:12:10 2019-10-21
           19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
           19:12:10 2019-10-21 19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
           19:12:10 2019-10-21 19:12:10.3153|INFO|Deploy.Program|File upload successfully done: yyy.json
           *{color:#ff0000}19:12:21 Agent went offline during the build19:12:21{color}* ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2838043[name=<node_name>]
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:629)19:12:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
           19:12:21 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)19:12:21 at java.util.concurrent.FutureTask.run(FutureTask.java:266)19:12:21 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           19:12:21 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           19:12:21 at java.lang.Thread.run(Thread.java:745)
           19:12:21 Caused by: java.io.IOException: Connection timed out19:12:21 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
           19:12:21 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
           19:12:21 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
           19:12:21 at sun.nio.ch.IOUtil.read(IOUtil.java:197)19:12:21 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
           19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:137)
           19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:310)
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
           19:12:21 ... 6 more19:12:21 
           * *Screenshot configuring the node*

           

          *I hope that someone can help me out this issue. When go DuckDuckGo than I find out I'm not the only one who have this issue but I can't find a solution.* 

          *Regards,*

          *Nico*
          The main job is started, which is starting a some other (sub)jobs. Mostly at the same sub-job, after the first four jobs has been executed without any problem, a job is started. When that job is started or if this job runs one or three minutes we get the issue "Agent went offline". When it runs like say 4 min. than it runs to the end without any issue! That job runs for 1,5 hour or even more!

          After it went wrong with "Agent went offliner" or with "Channel already closed" we start the main job again and than it works always without a issue!!!!????? WHY?????

          Analyzing this:
           * *Windows event viewer* (see screenshot), I can see that the Jenkinsslave-C_Jenkins is "normally" stopped. But by whom? The Job is not doing this or at least so far as I can see the Job is not doing this and should not do this. See my screenshot. In this screenshot it's a few seconds later the service is stopped. But I have other circumstances and then it was in the exact same second!
           * *Jenkins logging "Jenkins-slave.err"*. There I see issue's with JNLP connection but on this moment I don't know what to do about it! What do I have to change or were can I find documentation. 

          INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:01:43 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           *{color:#ff0000}Oct 21, 2019 7:12:21 PM hudson.remoting.jnlp.Main$CuiListener status{color}*
           *{color:#ff0000}INFO: Terminated{color}*
           Oct 21, 2019 7:12:31 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect
           INFO: Restarting agent via jenkins.slaves.restarter.WinswSlaveRestarter@48eae634
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main createEngine
           INFO: Setting up slave: nodename
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener <init>
           INFO: Jenkins agent is running in headless mode.
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Locating server among [http://&lt;ip adres>/jenkins/|http://%3Cip%20adres%3E/jenkins/]
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Handshaking
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:12:34 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main createEngine
           INFO: Setting up slave: nodename
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener <init>
           INFO: Jenkins agent is running in headless mode.
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Locating server among [http://&lt;ip adress>jenkins/|http://%3Cip%20adress%3Ejenkins/]
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Handshaking
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP3-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Server didn't accept the handshake: Unknown protocol:Protocol:JNLP3-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connecting to <ip adress>:<fixed port adress>
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Trying protocol: JNLP2-connect
           Oct 21, 2019 7:13:58 PM hudson.remoting.jnlp.Main$CuiListener status
           INFO: Connected
           Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.
           Oct 21, 2019 10:51:41 PM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           Oct 22, 2019 4:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           Oct 22, 2019 10:51:41 AM hudson.plugins.tmpcleaner.TmpCleanTask call
           INFO: end TmpCleanTask
           * *The console log* Agent went offline at the same time the JNLP is TERMINATED in Jenkins-slave.err file!:

          19:12:10 2019-10-21
           19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
           19:12:10 2019-10-21 19:12:10.2303|INFO|Deploy.Program|File upload successfully done: xxxx.json
           19:12:10 2019-10-21 19:12:10.3153|INFO|Deploy.Program|File upload successfully done: yyy.json
           *{color:#ff0000}19:12:21 Agent went offline during the build
          {color}*{color:#172b4d}19:12:21{color} ERROR: Connection was broken: java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2838043[name=<node_name>]
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:629)19:12:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
           19:12:21 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)19:12:21 at java.util.concurrent.FutureTask.run(FutureTask.java:266)19:12:21 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           19:12:21 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           19:12:21 at java.lang.Thread.run(Thread.java:745)
           19:12:21 Caused by: java.io.IOException: Connection timed out19:12:21 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
           19:12:21 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
           19:12:21 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
           19:12:21 at sun.nio.ch.IOUtil.read(IOUtil.java:197)19:12:21 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
           19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:137)
           19:12:21 at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:310)
           19:12:21 at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
           19:12:21 ... 6 more19:12:21 
           * *Screenshot configuring the node*

           

          *I hope that someone can help me out this issue. When go DuckDuckGo than I find out I'm not the only one who have this issue but I can't find a solution.* 

          *Regards,*

          *Nico*
          nkamp01 Nico van de Kamp made changes -
          Environment We have a Linux master with 77 windows 2012R2 slaves and a few linux node's.

          On this moment we have Jenkins 2.9. In the near future we will update Jenkins to the latest version and split up the environment into DEV, ACC and PRD. So than we have three masters with each approx. 25 windows 2012R2 nodes (slaves)
          We have a Linux master with 77 windows 2012R2 slaves and a few linux node's.

          On this moment we have Jenkins 2.9. In the near future we will update Jenkins to the latest version and split up the environment into DEV, ACC and PRD. So than we have three masters with each approx. 25 windows 2012R2 nodes (slaves)

          Under Manage Jenkins --> Globaal security we have enabled security and "TCP port for JNLP agents" is Fixed with a port number.
          Hide
          nkamp01 Nico van de Kamp added a comment -

          Ok, I understand now that, to take these  JVM options into account by the slave I do have to remove the slave first on the Windows server and then install that slave again. Than these options are taken into account by slave in the JNLP file. I was expecting that what I have filled in here, when I stop/start the Jenkins service on the slave, that these options are taken into account by the slave!

          I have not set up the environment, so this is new to me. I have created a jobs, and a lot of scripting but not configured the master and slaves. And I hope that we go asap to the latest Jenkins version but till than, I have this issue. And I don't know what to do than try a lot of things.

          Show
          nkamp01 Nico van de Kamp added a comment - Ok, I understand now that, to take these  JVM options into account by the slave I do have to remove the slave first on the Windows server and then install that slave again. Than these options are taken into account by slave in the JNLP file. I was expecting that what I have filled in here, when I stop/start the Jenkins service on the slave, that these options are taken into account by the slave! I have not set up the environment, so this is new to me. I have created a jobs, and a lot of scripting but not configured the master and slaves. And I hope that we go asap to the latest Jenkins version but till than, I have this issue. And I don't know what to do than try a lot of things.

            People

            Assignee:
            praqma Praqma Support
            Reporter:
            nkamp01 Nico van de Kamp
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: