Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69149

Git client "accept new host key" breaks SSH auth from OpenSSH 7.5 and earlier

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • git-client-plugin
    • None
    • Debian Linux 9.13
      OpenSSH 7.4p1
    • 3.11.2

      After updating jenkins to 2.361 and all plugins to latest versions, jenkins ssh attempts to clone gitlab repository (standard git client) and fails with output:

      Setting origin to git@gitlab.xxxxx:yyyy/project.git
      > git config remote.origin.url git@gitlab.xxxxx:yyyy/project.git # timeout=10
      Fetching origin...
      Fetching upstream changes from origin
      > git --version # timeout=10
      > git --version # 'git version 2.11.0'
      > git config --get remote.origin.url # timeout=10
      using GIT_SSH to set credentials Gitlab Jenkins SSH Key
      Verifying host key using known hosts file, will automatically accept unseen keys
      > git fetch --tags --progress – origin +refs/heads/:refs/remotes/origin/ # timeout=10
      hudson.plugins.git.GitException: Command "git fetch --tags --progress – origin +refs/heads/:refs/remotes/origin/" returned status code 128:
      stdout: 
      stderr: command-line line 0: unsupported option "accept-new".
      fatal: Could not read from remote repository.

       
      Manually executing "ssh -o StrictHostKeyChecking=accept-new gitlab" returns error. Substituting 'accept-new' with 'no' results in no error.

          [JENKINS-69149] Git client "accept new host key" breaks SSH auth from OpenSSH 7.5 and earlier

          markewaite - Thank you for the quick turn around. However, to work around the issue I updated the affected systems to Debian 10 which both fixed the issue and ensured we had vendor supported OS systems running on our infrastructure. I am unfortunately unable to test the PR.

          Jesse McCormick added a comment - markewaite - Thank you for the quick turn around. However, to work around the issue I updated the affected systems to Debian 10 which both fixed the issue and ensured we had vendor supported OS systems running on our infrastructure. I am unfortunately unable to test the PR.

          Mark Waite added a comment -

          Thanks koan00. Testing on a Debian 10 system would be a help for the project, since it would be checking that a system with a newer OpenSSH version was not harmed by the changes.

          Mark Waite added a comment - Thanks koan00 . Testing on a Debian 10 system would be a help for the project, since it would be checking that a system with a newer OpenSSH version was not harmed by the changes.

          markewaite I have just tested a new build (git-client-3.11.2-rc3139.978264ccc070.hpi) on Ubuntu 18.04 LTS which is based on Debian 10. It works well.

          Evgeny Seliverstov added a comment - markewaite I have just tested a new build (git-client-3.11.2-rc3139.978264ccc070.hpi) on Ubuntu 18.04 LTS which is based on Debian 10. It works well.

          Mark Waite added a comment -

          Thanks theirix!

          Mark Waite added a comment - Thanks theirix !

          Mark Roland added a comment - - edited

          I believe I'm experiencing this as well on CentOS 7, Jenkins 2.346.2 after upgrading one of the git/pipeline related plugins (git, git-client, git, github, pipeline-groovy-lib, scm-api). I can't easily downgrade git-client because it's part of a string of dependencies related to pipelines.

          I originally posted on https://issues.jenkins.io/browse/JENKINS-69207, but I think this is the correct issue I'm having.

          Thank you.

          P.S. markewaite Thank you for this plugin and your responsiveness.

          P.P.S. The comment from syol above was the simplest resolution for me.

          Mark Roland added a comment - - edited I believe I'm experiencing this as well on CentOS 7, Jenkins 2.346.2 after upgrading one of the git/pipeline related plugins (git, git-client, git, github, pipeline-groovy-lib, scm-api). I can't easily downgrade git-client because it's part of a string of dependencies related to pipelines. I originally posted on https://issues.jenkins.io/browse/JENKINS-69207 , but I think this is the correct issue I'm having. Thank you. P.S. markewaite Thank you for this plugin and your responsiveness. P.P.S. The comment from syol above was the simplest resolution for me.

          Mark Waite added a comment - - edited

          markroland if the log file contains a reference to accept-new, then this is the issue you're seeing. Use "Manage Jenkins" -> "Configure Global Security" -> "Git Host Key Verification Configuration" to change the configuration from "Accept first connection" to "Known hosts file". No need to downgrade the plugin. No need to restart the Jenkins controller.

          If that setting does not resolve it, then you can switch to "Manually provided keys" and provide the ssh host keys for the ssh servers you use.

          If that setting does not resolve it, then you can switch to "No verification" and accept that you've reintroduced the man in the middle attack vector.

          Another shameless plug from me that you'll have better results if you move to a different operating system as your time and your schedules allow. CentOS 7 is supported but the utilities included in CentOS 7 are very old versions, including command line git 1.8.3.

          Command line git has recently released 2.37.1 and has been releasing new minor versions roughly every 3 months. You're sacrificing performance by staying with that operating system.

          Mark Waite added a comment - - edited markroland if the log file contains a reference to accept-new , then this is the issue you're seeing. Use "Manage Jenkins" -> "Configure Global Security" -> "Git Host Key Verification Configuration" to change the configuration from "Accept first connection" to "Known hosts file". No need to downgrade the plugin. No need to restart the Jenkins controller. If that setting does not resolve it, then you can switch to "Manually provided keys" and provide the ssh host keys for the ssh servers you use. If that setting does not resolve it, then you can switch to "No verification" and accept that you've reintroduced the man in the middle attack vector. Another shameless plug from me that you'll have better results if you move to a different operating system as your time and your schedules allow. CentOS 7 is supported but the utilities included in CentOS 7 are very old versions, including command line git 1.8.3. Command line git has recently released 2.37.1 and has been releasing new minor versions roughly every 3 months. You're sacrificing performance by staying with that operating system.

          Mark Roland added a comment -

          Thank you and I agree. We've updated our production environments, but as you can imagine CI is lagging behind. 

          Mark Roland added a comment - Thank you and I agree. We've updated our production environments, but as you can imagine CI is lagging behind. 

          Nok A added a comment -

          markewaite  - Thanks for your detailed explanation. I have tested the latest git-client-3.11.2-rc3139.978264ccc070.hpi with my Amazon Linux 2, Jenkins 2.361, and the "Known hosts file" set in the Git Host Key Verification Configuration. This combination works great and I didn't need to downgrade any plugin.  These are my software versions installed on my Amazon Linux 2 EC2:

          • Amazon Linux 2: aws-cli/1.18.147 Python/2.7.18 Linux/5.10.130-118.517.amzn2.x86_64 botocore/1.18.6
          • SSH: OpenSSH_7.4p1, OpenSSL 1.0.2k-fips (26 Jan 2017)
          • Git: 2.37.1
          • Jenkins: 2.361
          • Git Client Plugin: 3.11.2-rc3139.978264ccc070
          • Use "Manage Jenkins" -> "Configure Global Security" -> "Git Host Key Verification Configuration" -> Change the configuration from "Accept first connection" to "Known hosts file"

          Thanks a lot!

          Nok A added a comment - markewaite   - Thanks for your detailed explanation. I have tested the latest git-client-3.11.2-rc3139.978264ccc070.hpi with my Amazon Linux 2, Jenkins 2.361, and the "Known hosts file" set in the Git Host Key Verification Configuration. This combination works great and I didn't need to downgrade any plugin.  These are my software versions installed on my Amazon Linux 2 EC2: Amazon Linux 2: aws-cli/1.18.147 Python/2.7.18 Linux/5.10.130-118.517.amzn2.x86_64 botocore/1.18.6 SSH: OpenSSH_7.4p1, OpenSSL 1.0.2k-fips (26 Jan 2017) Git: 2.37.1 Jenkins: 2.361 Git Client Plugin: 3.11.2-rc3139.978264ccc070 Use "Manage Jenkins" -> "Configure Global Security" -> "Git Host Key Verification Configuration" -> Change the configuration from "Accept first connection" to " Known hosts file " Thanks a lot!

          Mark Waite added a comment -

          Thanks nokyai

          Mark Waite added a comment - Thanks nokyai

          Mark Waite added a comment -

          Released as git client plugin 3.11.2

          Mark Waite added a comment - Released as git client plugin 3.11.2

            markewaite Mark Waite
            koan00 Jesse McCormick
            Votes:
            15 Vote for this issue
            Watchers:
            29 Start watching this issue

              Created:
              Updated:
              Resolved: