Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63222

Cannot use yet-another-docker-plugin JNLP agents in 2.248+

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • core
    • Jenkins 2.248
    • 2.276 and 2.277.1

      The fix for https://issues.jenkins-ci.org/browse/JENKINS-63014 introduced a regression bug. This fix was released in Jenkins version 2.248 and broke a feature used in e.g. yet-another-docker plugin.

      Our Jenkins master is exposed via multiple URLs:

      • primary URL: this is the publicly used URL for the Jenkins server, our users and some of the slave agents  only capable of reaching Jenkins master through this address
      • secondary URL: this is an internal URL (only reachable from the host machine that runs the Jenkins master), some of the slave agents are only capable of reaching Jenkins master through this address

      We're using yet-another-docker plugin with JNLP launcher to launch Docker slaves. It has an option called "Different Jenkins master URL". This option can be used to overwrite the primary URL (the one set in Jenkins global config). Since version 2.248 Jenkins doesn't respect this secondary URL and the slaves fail to connect to Jenkins master.

      Log from previous versions:

      su - jenkins -c java -jar slave.jar -jnlpUrl SECONDARY_URL:8080/computer/docker_cloud-22c714f1d0ba//slave-agent.jnlp -secret "some secret id"
      Jul 27, 2020 11:03:06 AM hudson.remoting.jnlp.Main createEngine
      INFO: Setting up agent: docker_cloud-22c714f1d0ba
      Jul 27, 2020 11:03:06 AM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Jenkins agent is running in headless mode.
      Jul 27, 2020 11:03:06 AM hudson.remoting.Engine startEngine
      INFO: Using Remoting version: 4.3
      Jul 27, 2020 11:03:06 AM hudson.remoting.Engine startEngine
      WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
      Jul 27, 2020 11:03:06 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among [PRIMARY_URL, SECONDARY_URL]
      Jul 27, 2020 11:03:07 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
      INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
      Jul 27, 2020 11:03:07 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Agent discovery successful Agent address: SECONDARY_URL Agent port: 39633 Identity: 58:af:30:c6:91:24:f0:ab:ba:3f:ba:a6:d9:41:85:6f
      Jul 27, 2020 11:03:07 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Handshaking Jul 27, 2020 11:03:07 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to SECONDARY_URL:39633
      Jul 27, 2020 11:03:07 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Trying protocol: JNLP4-connect
      Jul 27, 2020 11:03:07 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Remote identity confirmed: 58:af:30:c6:91:24:f0:ab:ba:3f:ba:a6:d9:41:85:6f
      Jul 27, 2020 11:03:09 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      
      

      Log from 2.248 version:

      su - jenkins -c java -jar slave.jar -jnlpUrl http://SECONDARY_URL:8080/computer/docker_cloud-13120c5169f2//slave-agent.jnlp -secret "some secret id"
      Jul 27, 2020 10:49:06 AM hudson.remoting.jnlp.Main createEngine
      INFO: Setting up agent: docker_cloud-13120c5169f2
      Jul 27, 2020 10:49:06 AM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Jenkins agent is running in headless mode.
      Jul 27, 2020 10:49:06 AM hudson.remoting.Engine startEngine
      INFO: Using Remoting version: 4.5
      Jul 27, 2020 10:49:06 AM hudson.remoting.Engine startEngine
      WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
      Jul 27, 2020 10:49:06 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among [PRIMARY_URL]
      Jul 27, 2020 10:49:06 AM hudson.remoting.jnlp.Main$CuiListener error
      SEVERE: Failed to connect to PRIMARY_URL/tcpSlaveAgentListener/: PRIMARY_URL
      

          [JENKINS-63222] Cannot use yet-another-docker-plugin JNLP agents in 2.248+

          Jesse Glick added a comment - - edited

          Sounds like a bug in yet-another-docker-plugin to me. Jenkins defines only one root URL. There is nothing inherently wrong with having agents connect to an internal interface—this is the normal case in the kubernetes plugin; but it cannot use the -jnlpUrl launcher mode in that case: rather, it should directly use the lower-level launcher mode.

          Jesse Glick added a comment - - edited Sounds like a bug in yet-another-docker-plugin to me. Jenkins defines only one root URL. There is nothing inherently wrong with having agents connect to an internal interface—this is the normal case in the kubernetes plugin; but it cannot use the -jnlpUrl launcher mode in that case: rather, it should directly use the lower-level launcher mode.

          Jesse Glick added a comment -

          This and this is the problem, I think. When the Jenkins URL option is overridden, the plugin uses an inappropriate form of the agent launcher which will first contact the official Jenkins URL, fail, and then fall back to an inferred URL.

          Note that this plugin is not hosted on @jenkinsci and has no issue tracker component.

          Jesse Glick added a comment - This and this is the problem, I think. When the Jenkins URL option is overridden, the plugin uses an inappropriate form of the agent launcher which will first contact the official Jenkins URL, fail, and then fall back to an inferred URL. Note that this plugin is not hosted on @jenkinsci and has no issue tracker component.

          I don't think JENKINS-63014 should necessarily be reverted, but:

          > rather, it should directly use the lower-level launcher mode

          What would be the configuration to be used in that case, or where could we find what is the right arguments to pass in this case?

          Jonathan Ballet added a comment - I don't think JENKINS-63014 should necessarily be reverted, but: > rather, it should directly use the lower-level launcher mode What would be the configuration to be used in that case, or where could we find what is the right arguments to pass in this case?

          Jesse Glick added a comment -

          Basically the opposite of https://github.com/jenkinsci/docker-inbound-agent/pull/129 so rather than

          java -jar agent.jar -jnlpUrl https://jenkins/computer/myagent/slave-agent.jnlp -secret xxx123
          

          run

          java -classpath agent.jar hudson.remoting.jnlp.Main -headless -url https://jenkins/ xxx123 myagent
          

          which is basically what that JNLP file will wind up doing. The difference is that prior to JENKINS-63014, http://jenkins.local/computer/myagent/slave-agent.jnlp would actually be the equivalent of

          java -classpath agent.jar hudson.remoting.jnlp.Main -headless -url https://jenkins/ -url http://jenkins.local/ xxx123 myagent
          

          where the first would fail and only the second would work. Bypassing the *.jnlp and directly running the hudson.remoting.jnlp.Main entrypoint allows you to specify the one URL you actually expect to work in this context.

          (The downside is that some miscellaneous arguments injected into the *.jnlp, such as -workDir when configured in JNLPLauncher, or for that matter -webSocket, need to be passed explicitly. The kubernetes plugin deals with this by using the jenkins/inbound-agent image whose entry point interprets several environment variables that the plugin sets in the container.)

          A bit confusing I admit, not made any clearer by the fact that the -jnlpUrl mode does not actually use JNLP or Java Web Start but just parses a subset of the JNLP file format; Jenkins rarely uses real JWS any more.

          Jesse Glick added a comment - Basically the opposite of https://github.com/jenkinsci/docker-inbound-agent/pull/129 so rather than java -jar agent.jar -jnlpUrl https://jenkins/computer/myagent/slave-agent.jnlp -secret xxx123 run java -classpath agent.jar hudson.remoting.jnlp.Main -headless -url https://jenkins/ xxx123 myagent which is basically what that JNLP file will wind up doing. The difference is that prior to JENKINS-63014 , http: //jenkins.local/computer/myagent/slave-agent.jnlp would actually be the equivalent of java -classpath agent.jar hudson.remoting.jnlp.Main -headless -url https://jenkins/ -url http://jenkins. local / xxx123 myagent where the first would fail and only the second would work. Bypassing the *.jnlp and directly running the hudson.remoting.jnlp.Main entrypoint allows you to specify the one URL you actually expect to work in this context. (The downside is that some miscellaneous arguments injected into the *.jnlp , such as -workDir when configured in JNLPLauncher , or for that matter -webSocket , need to be passed explicitly. The kubernetes plugin deals with this by using the jenkins/inbound-agent image whose entry point interprets several environment variables that the plugin sets in the container.) A bit confusing I admit, not made any clearer by the fact that the -jnlpUrl mode does not actually use JNLP or Java Web Start but just parses a subset of the JNLP file format; Jenkins rarely uses real JWS any more.

          jglick OK thanks, I'll be able to try that this evening (UTC+2) for my use-case (reported in JENKINS-63014).
          (That won't solve the this very issue though.)

          Jonathan Ballet added a comment - jglick OK thanks, I'll be able to try that this evening (UTC+2) for my use-case (reported in JENKINS-63014 ). (That won't solve the this very issue though.)

          Orsolya Kiss added a comment -

          Thank you all for the quick responses! Based on this conversation I think JENKINS-63014 should not be reverted.

          I created an issue in the yad plugin's repo, hoping the problem will be solved soon: https://github.com/KostyaSha/yet-another-docker-plugin/issues/288

          Orsolya Kiss added a comment - Thank you all for the quick responses! Based on this conversation I think JENKINS-63014  should not be reverted. I created an issue in the yad plugin's repo, hoping the problem will be solved soon:  https://github.com/KostyaSha/yet-another-docker-plugin/issues/288

          Sorry for the delay, but we implemented the fix suggested by jglick in https://issues.jenkins-ci.org/browse/JENKINS-63222#comment-394889 and it seems to work better.
          We are running 2.250 now.

          Jonathan Ballet added a comment - Sorry for the delay, but we implemented the fix suggested by jglick in https://issues.jenkins-ci.org/browse/JENKINS-63222#comment-394889 and it seems to work better. We are running 2.250 now.

          Courtney Lake added a comment -

          jglick we applied your solution but we are getting unknown client name when trying to connect. Do you know what the client name should be. We are using AWS Instances and our client name was i-********** and it used to work before this update.

          Courtney Lake added a comment - jglick we applied your solution but we are getting unknown client name when trying to connect. Do you know what the client name should be. We are using AWS Instances and our client name was i-********** and it used to work before this update.

          Jesse Glick added a comment -

          I am afraid I know little about this plugin and cannot help you with details.

          Jesse Glick added a comment - I am afraid I know little about this plugin and cannot help you with details.

          Mark Waite added a comment -

          Jenkins 2.276 includes a system property jenkins.agent.inboundUrl that can be used to run the agent from a separate URL

          Mark Waite added a comment - Jenkins 2.276 includes a system property jenkins.agent.inboundUrl that can be used to run the agent from a separate URL

            Unassigned Unassigned
            orsoly Orsolya Kiss
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: