-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
Jenkins - Linux sever
Slave- Windows slave- Windows 10 OS
JDK 8 Update 131
Hi oleg_nenashev,
We upgraded Jenkins v2.90 version. Still, we are facing channels closed exception.Can you please check and provide the solution to resolve this? Since this makes our CI environment unstable.
Environment:
Jenkins server: Linux machine
Slave: Windows slave. Windows 10 OS.
Error Details
Connection was broken: java.nio.channels.ClosedChannelException
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)
at hudson.remoting.Channel.close(Channel.java:1405)
at hudson.remoting.Channel.close(Channel.java:1358)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:737)
at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)
at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:655)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
- relates to
-
-
- Reopened
-
[JENKINS-48895] Channels closed exception after upgrade Jenkins version 2.90
Vadivel Natarajan
created issue -
Vadivel Natarajan
made changes -
| Component/s | New: remote-terminal-access-plugin [ 17321 ] | |
| Component/s | New: remoting [ 15489 ] | |
| Description |
Original:
Hi [~oleg_nenashev], We upgraded Jenkins v2.90 version. Still we are facing channels closed exception.Can you please check and update reason? *Environment:* Jenkins server: Linux machine Slave :Windows slave. Windows 10 OS. *Error Details* Connection was broken: java.nio.channels.*ClosedChannelException* at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311) at hudson.remoting.Channel.close(Channel.java:1405) at hudson.remoting.Channel.close(Channel.java:1358) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:737) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:655) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) |
New:
Hi [~oleg_nenashev], We upgraded Jenkins v2.90 version. Still, we are facing channels closed exception.Can you please check and provide the solution to resolve this? Since this makes our CI environment unstable. *Environment:* Jenkins server: Linux machine Slave: Windows slave. Windows 10 OS. *Error Details* Connection was broken: java.nio.channels.*ClosedChannelException* at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311) at hudson.remoting.Channel.close(Channel.java:1405) at hudson.remoting.Channel.close(Channel.java:1358) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:737) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:655) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) |
| Environment |
Original:
Jenkins - Linux sever Slave- Windows slave- Windows 10 OS |
New:
Jenkins - Linux sever Slave- Windows slave- Windows 10 OS JDK 8 Update 131 |
| Summary | Original: Channels closed exception | New: Channels closed exception after upgrade Jenkins version 2.90 |
Form which version have you upgraded BTW?
And which Remoting version are you using on agents?
We are running version 3.15 on both our Windows slaves. However, only one of them seem to be behaving this way. While there are some differences between them, I am not really sure what would be relevant. If it is of interest, perhaps it is possible to dump system properties in some way?
I am not sure what version we were running before 2.97, but I would guess on Jenkins v. 2.92 based on when we updated.
Fabian Sörensson
added a comment - - edited We are running version 3.15 on both our Windows slaves. However, only one of them seem to be behaving this way. While there are some differences between them, I am not really sure what would be relevant. If it is of interest, perhaps it is possible to dump system properties in some way?
I am not sure what version we were running before 2.97, but I would guess on Jenkins v. 2.92 based on when we updated. | Link | New: This issue relates to JENKINS-48865 [ JENKINS-48865 ] |
Sam Beckwith III
made changes -
| Attachment | New: image-2018-06-04-10-18-35-266.png [ 42775 ] |
Sam Beckwith III
made changes -
| Attachment | Original: image-2018-06-04-10-18-35-266.png [ 42775 ] |
Sam Beckwith III
added a comment - In hopes of increasing visibility on this issue, I would like to share that I too have agents close connection for various reasons, sometimes unknown and sometimes known. The unknown reasons I have no information on other than these agents run on MS Windows of various versions or run in Kubernetes using the official docker image on stable Kubernetes hosts.
The known reasons have to do with running agents again in Kubernetes with the same image but on preemptable hosts (hosts that can be taken down by the cloud platform whenever it needs space). Thus the agent is shutdown in the middle of an operation which cause the channel to close.
My concern in this JIRA is the unknown reasons for channel close and no information in logs.
We are using Jenkins Master 2.107.3 & agents using jenkins/jnlp-slave:3.19-1 but also occurred on older versions of both.
Sam Beckwith III
added a comment - In hopes of increasing visibility on this issue, I would like to share that I too have agents close connection for various reasons, sometimes unknown and sometimes known. The unknown reasons I have no information on other than these agents run on MS Windows of various versions or run in Kubernetes using the official docker image on stable Kubernetes hosts.
The known reasons have to do with running agents again in Kubernetes with the same image but on preemptable hosts (hosts that can be taken down by the cloud platform whenever it needs space). Thus the agent is shutdown in the middle of an operation which cause the channel to close.
My concern in this JIRA is the unknown reasons for channel close and no information in logs.
We are using Jenkins Master 2.107.3 & agents using jenkins/jnlp-slave:3.19-1 but also occurred on older versions of both. Fabian Sörensson
added a comment - I can mention that we are not having these problems anymore! I think in our case, our problems were solved by something so simple as fiddling with the Windows sleep/hibernate options...
Fabian Sörensson
added a comment - I can mention that we are not having these problems anymore! I think in our case, our problems were solved by something so simple as fiddling with the Windows sleep/hibernate options...
+1 on this: We're having exactly the same problem. Same error, same environment. However, we're running Jenkins 2.101, but we had the same problem when running Jenkins 2.97.
This is a major problem for us, since most of our longer Jenkins builds fail due to a connection timeout.