-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
Server:
Jenkins 2.263.4 installed on Debian Buster using http://pkg.jenkins.io/debian-stable and apt
java -version
openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-post-Debian-1deb10u2)
OpenJDK 64-Bit Server VM (build 11.0.9.1+1-post-Debian-1deb10u2, mixed mode, sharing)
Agent:
Windows 10.0.18362 Build 18362
java -version
openjdk version "11.0.10" 2021-01-19 LTS
OpenJDK Runtime Environment Zulu11.45+28-SA (build 11.0.10+9-LTS)
OpenJDK 64-Bit Server VM Zulu11.45+28-SA (build 11.0.10+9-LTS, mixed mode)
Server: Jenkins 2.263.4 installed on Debian Buster using http://pkg.jenkins.io/debian-stable and apt java -version openjdk version "11.0.9.1" 2020-11-04 OpenJDK Runtime Environment (build 11.0.9.1+1-post-Debian-1deb10u2) OpenJDK 64-Bit Server VM (build 11.0.9.1+1-post-Debian-1deb10u2, mixed mode, sharing) Agent: Windows 10.0.18362 Build 18362 java -version openjdk version "11.0.10" 2021-01-19 LTS OpenJDK Runtime Environment Zulu11.45+28-SA (build 11.0.10+9-LTS) OpenJDK 64-Bit Server VM Zulu11.45+28-SA (build 11.0.10+9-LTS, mixed mode)
I installed the jenkins package from http://pkg.jenkins.io/debian-stable on a fresh installed Debian Buster machine using apt. After the initial configuration with the recommended plugins I added a new Node and tried to connect this agent.
Initially all looks fine, and the output of the agent.jar process tells
INFO: Connected
But a view minutes later the agent.jar process ends with the exception as shown below:
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Remote closed connection without specifying reason
{{ at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)}}
{{ at hudson.remoting.Engine.innerRun(Engine.java:744)}}
{{ at hudson.remoting.Engine.run(Engine.java:519)}}
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Remote closed connection without specifying reason
{{ at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:440)}}
{{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)}}
{{ at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:288)}}
{{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)}}
{{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)}}
{{ at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)}}
{{ at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)}}
{{ at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)}}
{{ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)}}
{{ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)}}
{{ at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:118)}}
{{ at java.base/java.lang.Thread.run(Unknown Source)}}
{{ Suppressed: java.nio.channels.ClosedChannelException}}
{{ ... 7 more}}
Mõrz 02, 2021 2:00:31 NACHM. hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
{{ at hudson.remoting.Engine.onConnectionRejected(Engine.java:829)}}
{{ at hudson.remoting.Engine.innerRun(Engine.java:769)}}
{{ at hudson.remoting.Engine.run(Engine.java:519)}}
More agent.jar log output can be found in the attached Agent.log
On the Jenkins server side I can see the following exception in the log:
2021-03-02 13:00:31.137+0000 [id=322] WARNING o.j.r.p.i.SSLEngineFilterLayer#onRecv: [JNLP4-connect connection from jenkins.windows.node/10.119.64.14:60826]
java.lang.NullPointerException
{{ at jenkins.slaves.DefaultJnlpSlaveReceiver.afterProperties(DefaultJnlpSlaveReceiver.java:127)}}
{{ at org.jenkinsci.remoting.engine.JnlpConnectionState$2.invoke(JnlpConnectionState.java:394)}}
{{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:337)}}
{{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterProperties(JnlpConnectionState.java:391)}}
{{ at org.jenkinsci.remoting.engine.JnlpProtocol4Handler$Handler.onReceiveHeaders(JnlpProtocol4Handler.java:323)}}
{{ at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecv(ConnectionHeadersFilterLayer.java:196)}}
{{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
{{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369)}}
{{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117)}}
{{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
{{ at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)}}
{{ at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:160)}}
{{ at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:795)}}
{{ at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)}}
{{ at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)}}
{{ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)}}
{{ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)}}
{{ at java.base/java.lang.Thread.run(Thread.java:834)}}
2021-03-02 13:00:31.137+0000 [id=322] SEVERE o.j.r.p.impl.NIONetworkLayer#ready: [JNLP4-connect connection from jenkins.windows.node/10.119.64.14:60826] Uncaught NullPointerException
java.lang.NullPointerException
{{ at jenkins.slaves.DefaultJnlpSlaveReceiver.afterProperties(DefaultJnlpSlaveReceiver.java:127)}}
{{ at org.jenkinsci.remoting.engine.JnlpConnectionState$2.invoke(JnlpConnectionState.java:394)}}
{{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:337)}}
{{ at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterProperties(JnlpConnectionState.java:391)}}
{{ at org.jenkinsci.remoting.engine.JnlpProtocol4Handler$Handler.onReceiveHeaders(JnlpProtocol4Handler.java:323)}}
{{ at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecv(ConnectionHeadersFilterLayer.java:196)}}
{{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
{{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369)}}
{{ at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117)}}
{{ at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)}}
{{ at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)}}
{{ at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:160)}}
{{ at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:795)}}
{{ at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)}}
{{ at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)}}
{{ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)}}
{{ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)}}
{{ at java.base/java.lang.Thread.run(Thread.java:834)}}
More form the Servers log output can be found in Jenkins.log and Node.log.
From the timing of the events I assume the following behavior:
- Agent connects to server without issues
- Server and Agent communicate a bit and then Server get a Nullpointer exception and closes the TCP channel
- Agent tries to send data over the closed channel and sees that he is already disconnected
- Agent tries to reconnect, but even if there was the Nullpointer exception from 2), the server still thinks the Agent is connected and refuses the reconnect
- Agent crashes, since it can not connect the server
So the root cause in my opinion is that there is this Nullpointer exception on Jenkins Server Side, a few seconds after an Agent connects.
I can reproduce this behavior with different Agent PCs, and even reinstalled the Jenkins Package on my Debian machine twice.
The Windows PC was connected to a different Jenkins Server (also on Debian Buster) before. There it operated without any issues. To switch to the new server, I deleted the agent.jar, the workspace and the java cache in %temp% before downloading the new agent.jar from the new server. I also updated the jre on the Windows agent during troubleshooting (from jre8 to jre11) to match the main version of the java jre.
- duplicates
-
JENKINS-64510 [remoting] Issues with TCP agent reconnects
- Resolved