Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64992

Can not connect agents to Jenkins on debian

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Critical
    • remoting
    • None

    Description

      I installed the jenkins package from  http://pkg.jenkins.io/debian-stable on a fresh installed Debian Buster machine using apt. After the initial configuration with the recommended plugins I added a new Node and tried to connect this agent.

      Initially all looks fine, and the output of the agent.jar process tells

      INFO: Connected

      But a view minutes later the agent.jar process ends with the exception as shown below:

      INFO: Protocol JNLP4-connect encountered an unexpected exception
      java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Remote closed connection without specifying reason
      {{ at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)}}
      {{ at hudson.remoting.Engine.innerRun(Engine.java:744)}}
      {{ at hudson.remoting.Engine.run(Engine.java:519)}}
      Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Remote closed connection without specifying reason
      {{ at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:440)}}
      {{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)}}
      {{ at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:288)}}
      {{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)}}
      {{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)}}
      {{ at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)}}
      {{ at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)}}
      {{ at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)}}
      {{ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
      {{ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
      {{ at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:118)}}
      {{ at java.base/java.lang.Thread.run(Unknown Source)}}
      {{ Suppressed: java.nio.channels.ClosedChannelException}}
      {{ ... 7 more}}

      Mõrz 02, 2021 2:00:31 NACHM. hudson.remoting.jnlp.Main$CuiListener error
      SEVERE: The server rejected the connection: None of the protocols were accepted
      java.lang.Exception: The server rejected the connection: None of the protocols were accepted
      {{ at hudson.remoting.Engine.onConnectionRejected(Engine.java:829)}}
      {{ at hudson.remoting.Engine.innerRun(Engine.java:769)}}
      {{ at hudson.remoting.Engine.run(Engine.java:519)}}

      More agent.jar log output can be found in the attached Agent.log

      On the Jenkins server side I can see the following exception in the log:

      2021-03-02 13:00:31.137+0000 [id=322] WARNING o.j.r.p.i.SSLEngineFilterLayer#onRecv: [JNLP4-connect connection from jenkins.windows.node/10.119.64.14:60826]
      java.lang.NullPointerException
      {{ at jenkins.slaves.DefaultJnlpSlaveReceiver.afterProperties(DefaultJnlpSlaveReceiver.java:127)}}
      {{ at org.jenkinsci.remoting.engine.JnlpConnectionState$2.invoke(JnlpConnectionState.java:394)}}
      {{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:337)}}
      {{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterProperties(JnlpConnectionState.java:391)}}
      {{ at org.jenkinsci.remoting.engine.JnlpProtocol4Handler$Handler.onReceiveHeaders(JnlpProtocol4Handler.java:323)}}
      {{ at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecv(ConnectionHeadersFilterLayer.java:196)}}
      {{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
      {{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369)}}
      {{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117)}}
      {{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
      {{ at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)}}
      {{ at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:160)}}
      {{ at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:795)}}
      {{ at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)}}
      {{ at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)}}
      {{ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)}}
      {{ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)}}
      {{ at java.base/java.lang.Thread.run(Thread.java:834)}}
      2021-03-02 13:00:31.137+0000 [id=322] SEVERE o.j.r.p.impl.NIONetworkLayer#ready: [JNLP4-connect connection from jenkins.windows.node/10.119.64.14:60826] Uncaught NullPointerException
      java.lang.NullPointerException
      {{ at jenkins.slaves.DefaultJnlpSlaveReceiver.afterProperties(DefaultJnlpSlaveReceiver.java:127)}}
      {{ at org.jenkinsci.remoting.engine.JnlpConnectionState$2.invoke(JnlpConnectionState.java:394)}}
      {{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:337)}}
      {{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterProperties(JnlpConnectionState.java:391)}}
      {{ at org.jenkinsci.remoting.engine.JnlpProtocol4Handler$Handler.onReceiveHeaders(JnlpProtocol4Handler.java:323)}}
      {{ at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecv(ConnectionHeadersFilterLayer.java:196)}}
      {{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
      {{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369)}}
      {{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117)}}
      {{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
      {{ at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)}}
      {{ at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:160)}}
      {{ at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:795)}}
      {{ at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)}}
      {{ at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)}}
      {{ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)}}
      {{ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)}}
      {{ at java.base/java.lang.Thread.run(Thread.java:834)}}

      More form the Servers log output can be found in Jenkins.log and Node.log.

       

      From the timing of the events I assume the following behavior:

      1. Agent connects to server without issues
      2. Server and Agent communicate a bit and then Server get a Nullpointer exception and closes the TCP channel
      3. Agent tries to send data over the closed channel and sees that he is already disconnected
      4. Agent tries to reconnect, but even if there was the  Nullpointer exception from 2), the server still thinks the Agent is connected and refuses the reconnect
      5. Agent crashes, since it can not connect the server

      So the root cause in my opinion is that there is this Nullpointer exception on Jenkins Server Side, a few seconds after an Agent connects.

      I can reproduce this behavior with different Agent PCs, and even reinstalled the Jenkins Package on my Debian machine twice.

      The Windows PC was connected to a different Jenkins Server (also on Debian Buster) before. There it operated without any issues. To switch to the new server, I deleted the agent.jar, the workspace and the java cache in %temp% before downloading the new agent.jar from the new server. I also updated the jre on the Windows agent during troubleshooting (from jre8 to jre11) to match the main version of the java jre.

      Attachments

        1. Agent.47jar.log
          3 kB
        2. Agent.log
          5 kB
        3. Jenkins.47jar.log
          34 kB
        4. Jenkins.log
          4 kB
        5. Node.47jar.log
          3 kB
        6. Node.log
          6 kB

        Issue Links

          Activity

            People

              Unassigned Unassigned
              tobiaszellner Tobias
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: