Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68542

Broken reconnection when using websockets

    XMLWordPrintable

Details

    • 2.354, 2.346.1

    Description

      When using a websocket agent, it it gets disconnected, the reconnection attempt can fail with

      WARNING	o.e.j.s.h.ContextHandler$Context#log: Error while serving https://JENKINS_URL/MY_NODE/wsagents/
      org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: computer_ambiguous_widow is already connected to this controller. Rejecting this connection.
      	at jenkins.slaves.DefaultJnlpSlaveReceiver.afterProperties(DefaultJnlpSlaveReceiver.java:142)
      	at org.jenkinsci.remoting.engine.JnlpConnectionState.fire(JnlpConnectionState.java:337)
      	at org.jenkinsci.remoting.engine.JnlpConnectionState.fireAfterProperties(JnlpConnectionState.java:386)
      	at jenkins.agents.WebSocketAgents.doIndex(WebSocketAgents.java:96)
      	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710)
      	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:398)
      Caused: java.lang.reflect.InvocationTargetException
      	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:402)
      	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:410)
      	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:208)
      	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:141)
      	at org.kohsuke.stapler.IndexDispatcher.dispatch(IndexDispatcher.java:28)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
      	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:475)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:694)
      	at org.kohsuke.stapler.Stapler.service(Stapler.java:240)
      

      Looking at the existing code shows there is existing handling to disconnect a connection if the agent tries to reconnect with the same cookie
      https://github.com/jenkinsci/jenkins/blob/216e7544c33423176759bd225f2706b498488711/core/src/main/java/jenkins/slaves/DefaultJnlpSlaveReceiver.java#L129-L147

      This is not implemented for websockets at the moment. The server side delivers a random cookie https://github.com/jenkinsci/jenkins/blob/5f9100286be16c434e973efc453a0b35ceee9342/core/src/main/java/jenkins/agents/WebSocketAgents.java#L101 however the client doesn't use it https://github.com/jenkinsci/remoting/blob/27c65d6dc6167b21b08793d58b1245fceb8c83ed/src/main/java/hudson/remoting/Engine.java#L559

      The client should record the cookie and use it for reconnection attempts.

      Attachments

        Issue Links

          Activity

            vlatombe Vincent Latombe added a comment - - edited

            It doesn't seem to be possible to fix this only in remoting.

            The server-side implementation of websockets doesn't seem to capture the given cookie so when running this line both cookie and channelCookie are always null, and as a result it ends up rejecting any connection attempt if it thinks the connection is still live.

            vlatombe Vincent Latombe added a comment - - edited It doesn't seem to be possible to fix this only in remoting. The server-side implementation of websockets doesn't seem to capture the given cookie so when running this line both cookie and channelCookie are always null , and as a result it ends up rejecting any connection attempt if it thinks the connection is still live.
            jglick Jesse Glick added a comment -

            Did you find a way to reproduce this by the way?

            jglick Jesse Glick added a comment - Did you find a way to reproduce this by the way?

            I'm not able to reproduce the disconnection directly, I believe this is the result of a half-open connection, so the client sees the disconnect and attempts to reconnect, while the server still thinks it is connected.

            vlatombe Vincent Latombe added a comment - I'm not able to reproduce the disconnection directly, I believe this is the result of a half-open connection, so the client sees the disconnect and attempts to reconnect, while the server still thinks it is connected.

            People

              vlatombe Vincent Latombe
              vlatombe Vincent Latombe
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: