Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-74734

jenkins-cli.jar hangs when using ssh

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Minor Minor
    • cli
    • Jenkins 2.481
      (java version "17.0.13")

      When attempting to use the jenkins-cli.jar shipped with Jenkins 2.481 (or 2.480) over SSH to Jenkins 2.481 (or 2.480), it hangs.

      NOTE: The examples below are with Jenkins unlocked (no user/name password required). With Jenkins locked (user/name password required) and a public key added for this SSH connection, the same symptoms are experienced.

      Using the jenkins-cli.jar shipped with Jenkins 2.481 over SSH to Jenkins 2.481, it hangs:

      $ java -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://127.0.0.1:8080 -ssh -user "dummer_user" help
      Oct 24, 2024 9:53:11 AM io.jenkins.cli.shaded.org.apache.sshd.common.util.security.AbstractSecurityProviderRegistrar getOrCreateProvider
      INFO: getOrCreateProvider(EdDSA) created instance of io.jenkins.cli.shaded.net.i2p.crypto.eddsa.EdDSASecurityProvider
      Oct 24, 2024 9:53:11 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.DefaultIoServiceFactoryFactory getIoServiceProvider
      INFO: No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
      Oct 24, 2024 9:53:11 AM io.jenkins.cli.shaded.org.apache.sshd.client.config.hosts.ConfigFileHostEntryResolver resolveEffectiveResolver
      INFO: resolveEffectiveResolver(dummer_user@127.0.0.1:53801/null) no configuration file at /home/xxxx/.ssh/config

      Using the jenkins-cli.jar shipped with Jenkins 2.479 over SSH to Jenkins 2.481, it works:

      $ java -jar "jenkins-cli.jar" -s http://127.0.0.1:8080 -ssh -user "dummer_user" help
      Oct 24, 2024 9:55:53 AM io.jenkins.cli.shaded.org.apache.sshd.common.util.security.AbstractSecurityProviderRegistrar getOrCreateProvider
      INFO: getOrCreateProvider(EdDSA) created instance of io.jenkins.cli.shaded.net.i2p.crypto.eddsa.EdDSASecurityProvider
      Oct 24, 2024 9:55:53 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.DefaultIoServiceFactoryFactory getIoServiceProvider
      INFO: No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
      Oct 24, 2024 9:55:53 AM io.jenkins.cli.shaded.org.apache.sshd.client.config.hosts.ConfigFileHostEntryResolver resolveEffectiveResolver
      INFO: resolveEffectiveResolver(dummer_user@127.0.0.1:53801/null) no configuration file at /home/xxxx/.ssh/configOct 24, 2024 9:55:53 AM io.jenkins.cli.shaded.org.apache.sshd.client.keyverifier.DefaultKnownHostsServerKeyVerifier reloadKnownHosts
      WARNING: reloadKnownHosts(/home/xxxx/.ssh/known_hosts) invalid file permissions: Permissions violation (GROUP_WRITE)
      Oct 24, 2024 9:55:53 AM hudson.cli.SSHCLI$1 verifyServerKey
      WARNING: Unknown host key for /127.0.0.1:53801
      add-job-to-view
          Adds jobs to view.`
      build
      ~

      Hanging session with logger enabled; Using the jenkins-cli.jar shipped with Jenkins 2.481 over SSH to Jenkins 2.481:

      $ java -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://127.0.0.1:8080 -ssh -user "dummer_user" -logger ALL help
      Oct 24, 2024 10:01:21 AM hudson.cli.CLI _main
      FINE: using connection mode SSH
      Oct 24, 2024 10:01:21 AM hudson.cli.SSHCLI sshConnection
      FINE: Connecting via SSH to: 127.0.0.1:53801
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.util.security.SecurityUtils register
      FINE: register(BC) not registered - enabled=true, supported=false
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.util.security.AbstractSecurityProviderRegistrar getOrCreateProvider
      INFO: getOrCreateProvider(EdDSA) created instance of io.jenkins.cli.shaded.net.i2p.crypto.eddsa.EdDSASecurityProvider
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.helpers.AbstractFactoryManager addSessionListener
      FINEST: addSessionListener(SshClient[7bb58ca3])[io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.SessionTimeoutListener@78b1cc93] registered
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.util.threads.SshdThreadFactory newThread
      FINEST: newThread(java.lang.ThreadGroup[name=main,maxpri=10])[sshd-SshClient[7bb58ca3]-timer-thread-1] [runnable=java.util.concurrent.ThreadPoolExecutor$Worker@2aa5fe93State
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.DefaultIoServiceFactoryFactory getIoServiceProvider
      INFO: No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Service <init>
      FINEST: Creating Nio2Connector
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.client.config.hosts.ConfigFileHostEntryResolver resolveEffectiveResolver
      INFO: resolveEffectiveResolver(dummer_user@127.0.0.1:53801/null) no configuration file at /home/xxxx/.ssh/config
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.client.config.hosts.ConfigFileHostEntryResolver resolveEffectiveHost
      FINE: resolveEffectiveHost(dummer_user@127.0.0.1:53801/null) => null
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.client.SshClient resolveHost
      FINE: connect(dummer_user@127.0.0.1:53801) no overrides
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Connector connect
      FINE: Connecting to /127.0.0.1:53801
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Service setOption
      FINE: setOption(SO_REUSEADDR)[true] from property=Property[socket-reuseaddr](Boolean]
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.util.threads.SshdThreadFactory newThread
      FINEST: newThread(java.lang.ThreadGroup[name=main,maxpri=10])[sshd-SshClient[7bb58ca3]-nio2-resume-thread-1] [runnable=java.util.concurrent.ThreadPoolExecutor$Worker@60d8c9b7State
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Connector lambda$connect$1
      FINE: connect(/127.0.0.1:53801): waiting for connection (timeout=60000ms)
      Oct 24, 2024 10:01:22 AM io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Session <init>
      FINE: Creating IoSession on /127.0.0.1:34952 from /127.0.0.1:53801 via null

        1. run-jenkins.sh
          0.8 kB
          Mark Waite
        2. plugins.txt
          2 kB
          Mark Waite

          [JENKINS-74734] jenkins-cli.jar hangs when using ssh

          Mark Waite added a comment - - edited

          I'm unable to duplicate the issue as described. You'll need to provide more details so that others can duplicate the issue. "How to report an issue" includes a checklist for many of the items needed to allow others to duplicate an issue report.

          Steps that I took while trying to duplicate the issue:

          1. Create a plugins.txt file that lists the plugins and their versions
          2. Create a run-jenkins.sh file that downloads Jenkins 2.481 and downloads the plugins defined in plugins.txt and runs Jenkins
          3. Run the run-jenkins.sh file on Debian Linux and complete the setup wizard by installing no additional plugins. Configure a single user 'mwaite' with password 'mwaite'
          4. Enable the ssh server inside Jenkins on port 22022 from the "Manage Jenkins" -> "Security" page
          5. Define an RSA private key / public key pair with the command ssh-keygen -t rsa -f ~/.ssh/id_rsa
          6. Add the RSA public key to the 'mwaite' user account from the "/user/mwaite/security/" URL
          7. Download the Jenkins CLI jar file from "/jnlpJars/jenkins-cli.jar"
          8. Run the Jenkins CLI jar file with "java -jar jenkins-cli.jar -s http://testing-b.markwaite.net:8080/ -ssh -user mwaite help" and confirm that it responds as expected

          The ssh server inside Jenkins does not understand ECDSA or ED25519 public keys and will not recognize their use. The Jenkins CLI seems to rely on ~/.ssh/id_rsa as its private key. I was able to successfully authenticate to Jenkins CLI with ~/.ssh/id_rsa.

          Mark Waite added a comment - - edited I'm unable to duplicate the issue as described. You'll need to provide more details so that others can duplicate the issue. "How to report an issue" includes a checklist for many of the items needed to allow others to duplicate an issue report. Steps that I took while trying to duplicate the issue: Create a plugins.txt file that lists the plugins and their versions Create a run-jenkins.sh file that downloads Jenkins 2.481 and downloads the plugins defined in plugins.txt and runs Jenkins Run the run-jenkins.sh file on Debian Linux and complete the setup wizard by installing no additional plugins. Configure a single user 'mwaite' with password 'mwaite' Enable the ssh server inside Jenkins on port 22022 from the "Manage Jenkins" -> "Security" page Define an RSA private key / public key pair with the command ssh-keygen -t rsa -f ~/.ssh/id_rsa Add the RSA public key to the 'mwaite' user account from the "/user/mwaite/security/" URL Download the Jenkins CLI jar file from "/jnlpJars/jenkins-cli.jar" Run the Jenkins CLI jar file with "java -jar jenkins-cli.jar -s http://testing-b.markwaite.net:8080/ -ssh -user mwaite help" and confirm that it responds as expected The ssh server inside Jenkins does not understand ECDSA or ED25519 public keys and will not recognize their use. The Jenkins CLI seems to rely on ~/.ssh/id_rsa as its private key. I was able to successfully authenticate to Jenkins CLI with ~/.ssh/id_rsa .

          Steve added a comment -

          Thanks for investigating. I will come back to you with my exact steps to re create.

          Steve added a comment - Thanks for investigating. I will come back to you with my exact steps to re create.

          Mark Waite added a comment -

          Closing after a month with no further information to duplicate the issue.

          Mark Waite added a comment - Closing after a month with no further information to duplicate the issue.

            Unassigned Unassigned
            vsteve Steve
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: