Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28499

tcpSlaveAgentListener not found

    XMLWordPrintable

Details

    Description

      Upgraded from 1.599 to 1.614.
      JNLP slaves now fail to connect with following error, where <HOST> is my actual host.
      SSH slaves are unaffected.

      May 20, 2015 10:57:51 AM hudson.remoting.jnlp.Main$CuiListener error
      SEVERE: https://<HOST>/tcpSlaveAgentListener/ is invalid: 404 Not Found
      java.lang.Exception: https://<HOST>/tcpSlaveAgentListener/ is invalid: 404 Not Found
      at hudson.remoting.Engine.run(Engine.java:214)

      Attachments

        Issue Links

          Activity

            jameshowe James Howe added a comment -

            Downgraded to 1.611, and issue is resolved.

            jameshowe James Howe added a comment - Downgraded to 1.611, and issue is resolved.
            jameshowe James Howe added a comment -

            SSL is provided by an nginx proxy.

            jameshowe James Howe added a comment - SSL is provided by an nginx proxy.
            danielbeck Daniel Beck added a comment -

            Does 1.613 work correctly?

            danielbeck Daniel Beck added a comment - Does 1.613 work correctly?
            danielbeck Daniel Beck added a comment -

            Anything interesting in the Jenkins master log?

            danielbeck Daniel Beck added a comment - Anything interesting in the Jenkins master log?
            danielbeck Daniel Beck added a comment -

            No response to comment asking for additional information in several weeks, so resolving as Incomplete.

            Please reopen if this issue still occurs on the current Jenkins release, and provide the information requested in earlier comments. Also see https://wiki.jenkins-ci.org/display/JENKINS/How+to+report+an+issue for what information is typically useful in bug reports.

            danielbeck Daniel Beck added a comment - No response to comment asking for additional information in several weeks, so resolving as Incomplete. Please reopen if this issue still occurs on the current Jenkins release, and provide the information requested in earlier comments. Also see https://wiki.jenkins-ci.org/display/JENKINS/How+to+report+an+issue for what information is typically useful in bug reports.

            After an update this issue re-appeared, in our special configuration

            Jenkins 2.32.2
            Libvirt Plugin with 8 virtual machines at one and 4 times 2 virtual machines at 4 more real servers
            We suffer from issue 32118 (https://issues.jenkins-ci.org/browse/JENKINS-32118), therefore we added the following Java option at our startup script: \"-Dhudson.model.DirectoryBrowserSupport.CSP=sandbox allow-scripts; default-src 'none'; img-src 'self'; style-src 'self';\"

            After a restart, all our nodes stay offline (the virtual machines get started), because of this error on the clients

            starfire_ Andreas Tscharner added a comment - After an update this issue re-appeared, in our special configuration Jenkins 2.32.2 Libvirt Plugin with 8 virtual machines at one and 4 times 2 virtual machines at 4 more real servers We suffer from issue 32118 ( https://issues.jenkins-ci.org/browse/JENKINS-32118 ), therefore we added the following Java option at our startup script: \"-Dhudson.model.DirectoryBrowserSupport.CSP=sandbox allow-scripts; default-src 'none'; img-src 'self'; style-src 'self';\" After a restart, all our nodes stay offline (the virtual machines get started), because of this error on the clients

            Forget to add: If we enter this line into the script console, AFTER the startup of Jenkins, it works...

            starfire_ Andreas Tscharner added a comment - Forget to add: If we enter this line into the script console, AFTER the startup of Jenkins, it works...
            oleg_nenashev Oleg Nenashev added a comment -

            starfire_ Whatever happens in your use-case, it likely comes from this code: https://github.com/jenkinsci/jenkins/blob/496703d0fe133445e10c7d8d07fa7afd351c8854/core/src/main/java/jenkins/model/Jenkins.java#L1222-L1251

            Due to whatever reason, the tcpAgentListener gets initialized with null. Could you please provide the full startup command line? I would suspect that one option just messes up the another one related to the TcpAgentListener

            oleg_nenashev Oleg Nenashev added a comment - starfire_ Whatever happens in your use-case, it likely comes from this code: https://github.com/jenkinsci/jenkins/blob/496703d0fe133445e10c7d8d07fa7afd351c8854/core/src/main/java/jenkins/model/Jenkins.java#L1222-L1251 Due to whatever reason, the tcpAgentListener gets initialized with null. Could you please provide the full startup command line? I would suspect that one option just messes up the another one related to the TcpAgentListener

            It is the Debian startup script with the workaround:

            /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins --output=/var/log/jenkins/jenkins.log --pidfile=/var/run/jenkins/jenkins.pid – /usr/bin/java -Djava.awt.headless=true \"-Dhudson.model.DirectoryBrowserSupport.CSP=sandbox allow-scripts; default-src 'none'; img-src 'self'; style-src 'self';\" -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080

             

            I have also tried with:  \"-Dhudson.model.DirectoryBrowserSupport.CSP= \" to no avail.

             

            TIA and best regards - Andreas

            starfire_ Andreas Tscharner added a comment - It is the Debian startup script with the workaround: /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins --output=/var/log/jenkins/jenkins.log --pidfile=/var/run/jenkins/jenkins.pid – /usr/bin/java -Djava.awt.headless=true \"-Dhudson.model.DirectoryBrowserSupport.CSP=sandbox allow-scripts; default-src 'none'; img-src 'self'; style-src 'self';\" -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080   I have also tried with:  \"-Dhudson.model.DirectoryBrowserSupport.CSP= \" to no avail.   TIA and best regards - Andreas

            I've just realized that I have forgotten double quotes around the Java options:

            /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins --output=/var/log/jenkins/jenkins.log --pidfile=/var/run/jenkins/jenkins.pid – /usr/bin/java "-Djava.awt.headless=true \"-Dhudson.model.DirectoryBrowserSupport.CSP=sandbox allow-scripts; default-src 'none'; img-src 'self'; style-src 'self';\"" -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080

            starfire_ Andreas Tscharner added a comment - I've just realized that I have forgotten double quotes around the Java options: /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins --output=/var/log/jenkins/jenkins.log --pidfile=/var/run/jenkins/jenkins.pid – /usr/bin/java "-Djava.awt.headless=true \"-Dhudson.model.DirectoryBrowserSupport.CSP=sandbox allow-scripts; default-src 'none'; img-src 'self'; style-src 'self';\"" -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
            oleg_nenashev Oleg Nenashev added a comment -

            starfire_ is it working after that?

            oleg_nenashev Oleg Nenashev added a comment - starfire_ is it working after that?

            No. I was not clear enough before: I forgot the double quotes here in the issue tracker; on the system they have always been there...

            starfire_ Andreas Tscharner added a comment - No. I was not clear enough before: I forgot the double quotes here in the issue tracker; on the system they have always been there...
            oleg_nenashev Oleg Nenashev added a comment -

            I was pretty sure we have discussed it in this ticket, but maybe I messed up the things. Could you please share your Jenkins Global Security configuration? Is the JNLP port enabled there? If no, TCPAgentListener endpoint is missing

            oleg_nenashev Oleg Nenashev added a comment - I was pretty sure we have discussed it in this ticket, but maybe I messed up the things. Could you please share your Jenkins Global Security configuration? Is the JNLP port enabled there? If no, TCPAgentListener endpoint is missing
            christiancavegn Christian Cavegn added a comment - - edited

            Was this Issue ever resolved? I face exactly the same issue:

            • jenkins/jenkins:2.89.3 (same issue with v2.73.3) running on Kubernetes
            • jenkins/jnlp-slave:2.62

            I've configured Jenkins Global Security configuration / TCP port for JNLP agents to Fixed / 50000.
            When i start a new build, a new Slave is created but remains in state offline.

            When i exec into the slave and run jenkins-slave <secret> <slave-name> manually, i get the following info: 

            Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior
            Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main createEngine
            INFO: Setting up slave: default-jenkins-slave-5xbkp
            Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main$CuiListener <init>
            INFO: Jenkins agent is running in headless mode.
            Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main$CuiListener status
            INFO: Locating server among [http://jenkins-discovery.jenkins.svc.cluster.local:50000/]
            Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main$CuiListener error
            SEVERE: http://jenkins-discovery.jenkins.svc.cluster.local:50000/tcpSlaveAgentListener/ is invalid: 404 Not Found
            java.lang.Exception: http://jenkins-discovery.jenkins.svc.cluster.local:50000/tcpSlaveAgentListener/ is invalid: 404 Not Found
                    at hudson.remoting.Engine.run(Engine.java:244) 

            When i do a curl i get:

            curl http://jenkins-discovery.jenkins.svc.cluster.local:50000/
            Jenkins-Agent-Protocols: JNLP-connect, JNLP2-connect, JNLP4-connect, Ping
            Jenkins-Version: 2.89.2
            Jenkins-Session: 639cc7ba
            Client: xxx.xxx.xxx.xxx
            Server: xxx.xxx.xxx.xxx

            curl http://jenkins-discovery.jenkins.svc.cluster.local:50000/tcpSlaveAgentListener/
            Not Found

             -> Port 50000 is open and reachable, tcpSlaveAgentListener just isn't there

            christiancavegn Christian Cavegn added a comment - - edited Was this Issue ever resolved? I face exactly the same issue: jenkins/jenkins:2.89.3 (same issue with v2.73.3) running on Kubernetes jenkins/jnlp-slave:2.62 I've configured Jenkins Global Security configuration / TCP port for JNLP agents to Fixed / 50000. When i start a new build, a new Slave is created but remains in state offline. When i exec into the slave and run jenkins-slave <secret> <slave-name> manually, i get the following info:  Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: default-jenkins-slave-5xbkp Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Jenkins agent is running in headless mode. Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [ http://jenkins-discovery.jenkins.svc.cluster.local:50000/ ] Dec 25, 2017 11:57:44 AM hudson.remoting.jnlp.Main$CuiListener error SEVERE:  http://jenkins-discovery.jenkins.svc.cluster.local:50000/tcpSlaveAgentListener/  is invalid: 404 Not Found java.lang.Exception:  http://jenkins-discovery.jenkins.svc.cluster.local:50000/tcpSlaveAgentListener/  is invalid: 404 Not Found         at hudson.remoting.Engine.run(Engine.java:244)   When i do a curl i get: curl  http://jenkins-discovery.jenkins.svc.cluster.local:50000/ Jenkins-Agent-Protocols: JNLP-connect, JNLP2-connect, JNLP4-connect, Ping Jenkins-Version: 2.89.2 Jenkins-Session: 639cc7ba Client: xxx.xxx.xxx.xxx Server: xxx.xxx.xxx.xxx curl  http://jenkins-discovery.jenkins.svc.cluster.local:50000/tcpSlaveAgentListener/ Not Found  -> Port 50000 is open and reachable, tcpSlaveAgentListener just isn't there
            naensma Nacho Llorens added a comment -

            Same thing here. Is there a workaround for this?

            naensma Nacho Llorens added a comment - Same thing here. Is there a workaround for this?
            oleg_nenashev Oleg Nenashev added a comment -

            We have improved the thread failover in 2.85: JENKINS-38711. naensma christiancavegn are you running with 2.89.x LTS?

            oleg_nenashev Oleg Nenashev added a comment - We have improved the thread failover in 2.85: JENKINS-38711 . naensma christiancavegn are you running with 2.89.x LTS?
            naensma Nacho Llorens added a comment -

            Yes, I tried with 2.89.x lts docker image.

            naensma Nacho Llorens added a comment - Yes, I tried with 2.89.x lts docker image.
            oleg_nenashev Oleg Nenashev added a comment -

            naensma please check system logs. If the thread dies after the Jenkins startup, there should be always a message

            oleg_nenashev Oleg Nenashev added a comment - naensma please check system logs. If the thread dies after the Jenkins startup, there should be always a message
            naensma Nacho Llorens added a comment - - edited

            oleg_nenashev I dont see any stack trace in the System logs when Jenkins start.

            Everything seems fine. The config has port assigned in the security config page but the page is still not found.

            Is there a somewhat recent version I can workaround with?

            naensma Nacho Llorens added a comment - - edited oleg_nenashev I dont see any stack trace in the System logs when Jenkins start. Everything seems fine. The config has port assigned in the security config page but the page is still not found. Is there a somewhat recent version I can workaround with?

            oleg_nenashev Yes, i'm running LTS. I just saw that i entered a wrong version number, it's 2.89.2 of course, sorry about that.

            Please tell me if i shoud try another version / how i can provide additional information.

            christiancavegn Christian Cavegn added a comment - oleg_nenashev  Yes, i'm running LTS. I just saw that i entered a wrong version number, it's 2.89.2 of course, sorry about that. Please tell me if i shoud try another version / how i can provide additional information.

            oleg_nenashev naensma I think i just found some kind of solution:

            When i enter my connection string as 'Jenkins Tunnel' instead of 'Jenkins URL', my clients seem to be able to connect to the master.

            Does this make any sense?

            christiancavegn Christian Cavegn added a comment - oleg_nenashev naensma I think i just found some kind of solution: When i enter my connection string as 'Jenkins Tunnel' instead of 'Jenkins URL', my clients seem to be able to connect to the master. Does this make any sense?

            I am hitting the same issue. 

            Background

            Jenkins is running in an on premise kubernetes cluster.

            NAME     READY     STATUS    RESTARTS   AGE     IP                NODE 
            jenkins-0 1/1          Running   0                 5s        10.43.0.2     nem-docker-app-node05.inter-olymp.local

            There are two service exposed as NodePort. One for jenkins using NodePort 30100 and one for the slave port using 30200, which is set to 50000

            NAME                   TYPE                 CLUSTER-IP           EXTERNAL-IP    PORT(S)                    AGE     SELECTOR
            jenkins                  NodePort         10.96.191.11          <none>            8080:30100/TCP       8s        app=jenkins
            jenkins-agents      NodePort         10.108.78.226        <none>            50000:30200/TCP     8s        app=jenkins

            I browse to my Jenkins instance using port 30100, e.g. http://build.mydomain.com:30100 and create a new permanent agent called test.

            What I Expected

            Execute the following command and the slave agent connects to Jenkins.

            java -jar agent.jar -jnlpUrl http://build.mydomain.com:30100/computer/test/slave-agent.jnlp -secret c80dc7812095e908d9807b83dbb7406f03f8274f83f70ca407e9fcec2665ed7a -workDir "C:\Jenkins"

            What Happened

            I get a 404 not found.

            Oct 19, 2018 9:57:45 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
            INFO: Using C:\Jenkins\remoting as a remoting work directory
            Both error and output logs will be printed to C:\Jenkins\remoting
            Failing to obtain http://build.mydomain.com:30100/computer/test/slave-agent.jnlp?encrypt=true
            java.io.IOException: Failed to load http://build.mydomain.com:30100/computer/test/slave-agent.jnlp?encrypt=true: 404 Not Found
            at hudson.remoting.Launcher.parseJnlpArguments(Launcher.java:496)
            at hudson.remoting.Launcher.run(Launcher.java:322)
            at hudson.remoting.Launcher.main(Launcher.java:283)

            Additional Info

            Browsing to http://build.mydomain.com:30200>/ gives

            Jenkins-Agent-Protocols: JNLP4-connect, Ping
            Jenkins-Version: 2.121.3
            Jenkins-Session: bb58e25a
            Client: 10.42.0.0
            Server: 10.43.0.2
            Remoting-Minimum-Version: 2.60

            So I am not sure what is going on here. The agent port is reachable through NodePort 30200. It sends traffic to 50000. But the slave-agent.jnlp is not found

            nemligtim Timothy Harris added a comment - I am hitting the same issue.  Background Jenkins is running in an on premise kubernetes cluster. NAME     READY     STATUS    RESTARTS   AGE     IP                NODE  jenkins-0 1/1          Running   0                 5s        10.43.0.2     nem-docker-app-node05.inter-olymp.local There are two service exposed as NodePort . One for jenkins using NodePort 30100 and one for the slave port using 30200, which is set to 50000 NAME                   TYPE                 CLUSTER-IP           EXTERNAL-IP    PORT(S)                    AGE     SELECTOR jenkins                  NodePort         10.96.191.11          <none>            8080:30100/TCP       8s        app=jenkins jenkins-agents      NodePort         10.108.78.226        <none>            50000:30200/TCP     8s        app=jenkins I browse to my Jenkins instance using port 30100, e.g. http://build.mydomain.com:30100  and create a new permanent agent called test. What I Expected Execute the following command and the slave agent connects to Jenkins. java -jar agent.jar -jnlpUrl http://build.mydomain.com:30100/computer/test/slave-agent.jnlp -secret c80dc7812095e908d9807b83dbb7406f03f8274f83f70ca407e9fcec2665ed7a -workDir "C:\Jenkins" What Happened I get a 404 not found. Oct 19, 2018 9:57:45 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using C:\Jenkins\remoting as a remoting work directory Both error and output logs will be printed to C:\Jenkins\remoting Failing to obtain http://build.mydomain.com:30100/computer/test/slave-agent.jnlp?encrypt=true java.io.IOException: Failed to load http://build.mydomain.com:30100/computer/test/slave-agent.jnlp?encrypt=true: 404 Not Found at hudson.remoting.Launcher.parseJnlpArguments(Launcher.java:496) at hudson.remoting.Launcher.run(Launcher.java:322) at hudson.remoting.Launcher.main(Launcher.java:283) Additional Info Browsing to  http://build.mydomain.com :30200>/ gives Jenkins-Agent-Protocols: JNLP4-connect, Ping Jenkins-Version: 2.121.3 Jenkins-Session: bb58e25a Client: 10.42.0.0 Server: 10.43.0.2 Remoting-Minimum-Version: 2.60 So I am not sure what is going on here. The agent port is reachable through NodePort 30200. It sends traffic to 50000. But the slave-agent.jnlp is not found
            oleg_nenashev Oleg Nenashev added a comment -

            JENKINS-53461 is Jenkins Remoting should address this issue for Cloud-Native platforms by removing reliance on the TCP endpoint

            oleg_nenashev Oleg Nenashev added a comment - JENKINS-53461 is Jenkins Remoting should address this issue for Cloud-Native platforms by removing reliance on the TCP endpoint
            billwang Bill Wang added a comment - - edited

            I hit the same issue, but finally fixed because I hit the wrong port

            there are two default ports: 8080 and 50000 

            50000 is jnlp port, but when you connect to tcpSlaveAgentListener, you should use 8080

             curl localhost:8080/tcpSlaveAgentListener/

             

            billwang Bill Wang added a comment - - edited I hit the same issue, but finally fixed because I hit the wrong port there are two default ports: 8080 and 50000  50000 is jnlp port, but when you connect to  tcpSlaveAgentListener , you should use 8080 curl localhost:8080/tcpSlaveAgentListener/  

            People

              Unassigned Unassigned
              jameshowe James Howe
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: