Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48754

Jenkins Slaves do not connect after update from version 2.95 to version 2.99

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Blocker Blocker
    • core, remoting
    • None

      After having installed Jenkins Version 2.99, none of my slaves did connect to the server anymore. Even a restart of the java script on the slave did not help.

      Workaround: Update Remoting on the agent side to 3.15

          [JENKINS-48754] Jenkins Slaves do not connect after update from version 2.95 to version 2.99

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          content/_data/changelogs/weekly.yml
          http://jenkins-ci.org/commit/jenkins.io/ac42d83bf51e56e826991839a5471eccdd767543
          Log:
          Changelog: noting JENKINS-48761/JENKINS-48754 in Jenkins 2.100.

          The PR also adds warning banners to 2.98/2.99

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: content/_data/changelogs/weekly.yml http://jenkins-ci.org/commit/jenkins.io/ac42d83bf51e56e826991839a5471eccdd767543 Log: Changelog: noting JENKINS-48761 / JENKINS-48754 in Jenkins 2.100. The PR also adds warning banners to 2.98/2.99

          Code changed in jenkins
          User: R. Tyler Croy
          Path:
          content/_data/changelogs/weekly.yml
          http://jenkins-ci.org/commit/jenkins.io/99fe6d097fd1cc1f6797ad7b816c98dec3261a37
          Log:
          Merge pull request #1305 from oleg-nenashev/changelog/2.100

          Changelog: noting JENKINS-48761/JENKINS-48754 in Jenkins 2.100.

          Compare: https://github.com/jenkins-infra/jenkins.io/compare/62362762ff55...99fe6d097fd1

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/_data/changelogs/weekly.yml http://jenkins-ci.org/commit/jenkins.io/99fe6d097fd1cc1f6797ad7b816c98dec3261a37 Log: Merge pull request #1305 from oleg-nenashev/changelog/2.100 Changelog: noting JENKINS-48761 / JENKINS-48754 in Jenkins 2.100. Compare: https://github.com/jenkins-infra/jenkins.io/compare/62362762ff55...99fe6d097fd1

          Hi,

          Thank you for your actions. I have updated to the 2.100 version, restarted Jenkins, then go to my slave and refreshed my browser cache, finally I launch again the agent from the browser and unfortunally the result was identical. It doesn't work. I also tried to download the agent.jar and launch by command line, we have exactly the same thing. Same stack trace on the agent side...

          I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?

           

          However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed). 

           

          Thank you in advance for your help. 

           

          Regards,

          Bruno.

          Bruno Laoueillé added a comment - Hi, Thank you for your actions. I have updated to the 2.100 version, restarted Jenkins, then go to my slave and refreshed my browser cache, finally I launch again the agent from the browser and unfortunally the result was identical. It doesn't work. I also tried to download the agent.jar and launch by command line, we have exactly the same thing. Same stack trace on the agent side... I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?   However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed).    Thank you in advance for your help.    Regards, Bruno.

          Oleg Nenashev added a comment - - edited

          Thanks for the quick reply, digging in.

          > I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ?

          It's not. The version should be 3.15, maybe it's just a typo. But yes, all changes in 2.100 were on the master side, Remoting has not been changed.

          > However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed).

          I am not sure the version rollback from Jenkins UI operates correctly in Docker.
          My recommendation would be to replace a container.

          Thanks for the pointers regarding Docker though. Do you use the standard Jenkins image or your custom one?

          Oleg Nenashev added a comment - - edited Thanks for the quick reply, digging in. > I edited the downloaded agent.jar, it was still in 3.5 version, is it normal ? It's not. The version should be 3.15, maybe it's just a typo. But yes, all changes in 2.100 were on the master side, Remoting has not been changed. > However we are totally blocked without the possibility to come back to the 2.94 version (we can only come back to 2.99), do you think we have a possibility to come back to this version through the web interface (because we have no access to the docker administration where Jenkins is installed). I am not sure the version rollback from Jenkins UI operates correctly in Docker. My recommendation would be to replace a container. Thanks for the pointers regarding Docker though. Do you use the standard Jenkins image or your custom one?

          Bruno Laoueillé added a comment - - edited

          For the version 3.5 it's my bad, sorry, it's the 3.15 (bad typing).

           

          EDIT: For the Docker image, I dont know sorry, it's not me who managed it. I can ask to another person.

          Bruno Laoueillé added a comment - - edited For the version 3.5 it's my bad, sorry, it's the 3.15 (bad typing).   EDIT: For the Docker image, I dont know sorry, it's not me who managed it. I can ask to another person.

          Oleg Nenashev added a comment - - edited

          OK, Docker... So far I can say the following:

          • Master runs on the 8082 port according to the bundle, but the web UI suggests the 10004 port. It is a whatever port mapping or external proxy, I'd guess
          • Agent fails to connect to TcpAgentListener port, the connection is refused. Probably it's just because the port is not exposed
          • TcpAgentListener runs with port set to 0. It means the it will be taking a random port every restart
          • Since you run the master in Docker, I am not sure how it is supposed to work. You would have to expose each port on the Docker host. I doubt you do that.

          Please check which port is exposed up for TcpAgentListener, and make sure the same port is configured in the security global config. Since you run in Docker, I would definitely recommend setting port flags so that the port won't be changeable from UI:

          -Djenkins.model.Jenkins.slaveAgentPort=${YOUR_EXPOSED_PORT} -Djenkins.model.Jenkins.slaveAgentPortEnforce=true
          

          Example for Docker

          "TCP agent listener port=0" id=86 (0x56) state=RUNNABLE cpu=0% (running in native)
              at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
              at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
              at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
              at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)
          

          Oleg Nenashev added a comment - - edited OK, Docker... So far I can say the following: Master runs on the 8082 port according to the bundle, but the web UI suggests the 10004 port. It is a whatever port mapping or external proxy, I'd guess Agent fails to connect to TcpAgentListener port, the connection is refused. Probably it's just because the port is not exposed TcpAgentListener runs with port set to 0 . It means the it will be taking a random port every restart Since you run the master in Docker, I am not sure how it is supposed to work. You would have to expose each port on the Docker host. I doubt you do that. Please check which port is exposed up for TcpAgentListener, and make sure the same port is configured in the security global config. Since you run in Docker, I would definitely recommend setting port flags so that the port won't be changeable from UI: -Djenkins.model.Jenkins.slaveAgentPort=${YOUR_EXPOSED_PORT} -Djenkins.model.Jenkins.slaveAgentPortEnforce=true Example for Docker "TCP agent listener port=0" id=86 (0x56) state=RUNNABLE cpu=0% (running in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:157)

          Yes it's ok for that, moreover the configuration has not changed on that. We have put the 8081 port for Tcp agents with only JNLP/4 and it is accessible and configured on our Docker. (It was working well before updates).

          Bruno Laoueillé added a comment - Yes it's ok for that, moreover the configuration has not changed on that. We have put the 8081 port for Tcp agents with only JNLP/4 and it is accessible and configured on our Docker. (It was working well before updates).

          Oleg Nenashev added a comment -

          Back to this story...

          > We have put the 8081 port

          How did you do that? The threaddump definitely suggests you're running with a random port

          Oleg Nenashev added a comment - Back to this story... > We have put the 8081 port How did you do that? The threaddump definitely suggests you're running with a random port

          Hi,

          Sorry for late answer. But you are right, the configuration of the connection through tuneling was not set for the slave. I forgot it after my second installation (it was set only in the global security part).

          So after correcting this part the new version has worked perfectly.

          Thank you for your support.

          Regards,

          Bruno.

          Bruno Laoueillé added a comment - Hi, Sorry for late answer. But you are right, the configuration of the connection through tuneling was not set for the slave. I forgot it after my second installation (it was set only in the global security part). So after correcting this part the new version has worked perfectly. Thank you for your support. Regards, Bruno.

          Oleg Nenashev added a comment -

          Thanks for the reply! So this is not a defect. I will close it as a duplicate of JENKINS-48761, because the most of the voters/watchers followed the summary of the issue && timing was very coincidental

          Oleg Nenashev added a comment - Thanks for the reply! So this is not a defect. I will close it as a duplicate of JENKINS-48761 , because the most of the voters/watchers followed the summary of the issue && timing was very coincidental

            oleg_nenashev Oleg Nenashev
            michaelmotteler Michael M.
            Votes:
            4 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: