Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69484

Connection with WebSockets breaks after some time

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • Jenkins version 2365

      After upgrading from 2360 our Windows slaves connecting via WebSockets have an instable connection (I did not try it for Linux slaves).
      Windows slave are using Java 11.0.15.1

      After some "Read side closed" information (while no job is running) a "Failed to synchronize IO streams ..." is noticed while a job is running.
      Such a connection break causes a failed job.

      The slave connects via 

      curl -sO https://my-jenkins-master/jenkins/jnlpJars/agent.jar
      java -jar agent.jar -jnlpUrl https://my-jenkins-master/jenkins/computer/myslave/jenkins-agent.jnlp -secret ############# -workDir "c:\Jenkins"

       

      The log on the slave contains the following:

      Aug. 30, 2022 9:46:54 VORM. hudson.remoting.jnlp.Main createEngine
      INFO: Setting up agent: myslave
      Aug. 30, 2022 9:46:54 VORM. hudson.remoting.Engine startEngine
      INFO: Using Remoting version: 3046.v38db_38a_b_7a_86
      Aug. 30, 2022 9:46:54 VORM. org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
      INFO: Using c:\remoting as a remoting work directory
      Aug. 30, 2022 9:46:55 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: WebSocket connection open
      Aug. 30, 2022 9:46:55 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Read side closed
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Read side closed
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Terminated
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Performing onReconnect operation.
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: onReconnect operation completed.
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: WebSocket connection open
      Aug. 30, 2022 9:50:40 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Aug. 30, 2022 10:13:26 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Read side closed
      Aug. 30, 2022 10:13:26 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Read side closed
      Aug. 30, 2022 10:13:26 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Terminated
      Aug. 30, 2022 10:13:27 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Performing onReconnect operation.
      Aug. 30, 2022 10:13:27 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: onReconnect operation completed.
      Aug. 30, 2022 10:13:27 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: WebSocket connection open
      Aug. 30, 2022 10:13:27 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      Aug. 30, 2022 10:21:48 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Read side closed
      Aug. 30, 2022 10:21:53 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Read side closed
      Aug. 30, 2022 10:21:57 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Terminated
      Aug. 30, 2022 10:21:58 VORM. hudson.Launcher$RemoteLaunchCallable$1 join
      INFO: Failed to synchronize IO streams on the channel hudson.remoting.Channel@54917ae3:myslave
      hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@54917ae3:myslave": Remote call on myslave failed. The channel is closing down or has closed down
          at hudson.remoting.Channel.call(Channel.java:993)
          at hudson.remoting.Channel.syncIO(Channel.java:1732)
          at hudson.Launcher$RemoteLaunchCallable$1.join(Launcher.java:1411)
          at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.base/java.lang.reflect.Method.invoke(Method.java:566)
          at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:924)
          at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:902)
          at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
          at hudson.remoting.UserRequest.perform(UserRequest.java:211)
          at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          at hudson.remoting.Request$2.run(Request.java:376)
          at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:125)
          at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@54917ae3:myslave": channel is already closed
          at hudson.remoting.Engine$1AgentEndpoint.onClose(Engine.java:628)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusEndpointWrapper.onClose(TyrusEndpointWrapper.java:1241)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusWebSocket.onClose(TyrusWebSocket.java:110)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.ProtocolHandler.close(ProtocolHandler.java:481)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusWebSocket.close(TyrusWebSocket.java:244)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusWebSocket.close(TyrusWebSocket.java:254)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusRemoteEndpoint.close(TyrusRemoteEndpoint.java:480)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusSession.close(TyrusSession.java:220)
          at hudson.remoting.Engine$1AgentEndpoint$Transport.closeRead(Engine.java:661)
          at hudson.remoting.Channel.terminate(Channel.java:1074)
          at hudson.remoting.Channel$1.terminate(Channel.java:620)
          at hudson.remoting.AbstractByteBufferCommandTransport.terminate(AbstractByteBufferCommandTransport.java:314)
          at hudson.remoting.Engine$1AgentEndpoint.onClose(Engine.java:628)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusEndpointWrapper.onClose(TyrusEndpointWrapper.java:1241)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.TyrusWebSocket.onClose(TyrusWebSocket.java:110)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.frame.CloseFrame.respond(CloseFrame.java:112)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.core.ProtocolHandler.process(ProtocolHandler.java:819)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.client.TyrusClientEngine$TyrusReadHandler.handle(TyrusClientEngine.java:726)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.ClientFilter.processRead(ClientFilter.java:204)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:111)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:113)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.SslFilter.handleRead(SslFilter.java:402)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.SslFilter.processRead(SslFilter.java:365)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:111)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:113)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:295)
          at io.jenkins.remoting.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:279)
          at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127)
          at java.base/sun.nio.ch.Invoker$2.run(Invoker.java:219)
          at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          ... 1 more

      Aug. 30, 2022 10:22:00 VORM. hudson.remoting.Request$2 run
      INFO: Failed to send back a reply to the request UserRequest:UserRPCRequest:hudson.Launcher$RemoteProcess.join[](6): hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@54917ae3:myslave": channel is already closed
      Aug. 30, 2022 10:22:02 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Performing onReconnect operation.
      Aug. 30, 2022 10:22:02 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: onReconnect operation completed.
      Aug. 30, 2022 10:22:02 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: WebSocket connection open
      Aug. 30, 2022 10:22:02 VORM. hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected

       

      After falling back to 2360 (including all plug-ins) the issue is gone.

       

            Unassigned Unassigned
            thomasmerk Thomas Merk
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: