Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33021

trilead ssh MAC and key exchange algorithms severely outdated

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: ssh-slaves-plugin
    • Labels:
      None
    • Environment:
      Jenkins 1.647, ssh-slaves-plugin 1.10
    • Similar Issues:

      Description

      The supported macs and kex methods in trilead are severely outdated, resulting in connection issues with properly secured ssh daemons on target machines. For instance:

      sshd[9800]: fatal: no matching mac found: client hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5 server hmac-sha2-256,hmac-sha2-512,umac-64@openssh.com,hmac-ripemd160 [preauth]
      

      In JENKINS-14709 a suggestion is made to replace trilead with orion, but Orion is not being maintained either. Orion refers to Ganymed, but even that hasn't been looked at for almost 2 years: Ganymed commits. It does seem to support hmac-sha2 macs though.

      From JENKINS-36873 (dupe)

      The ssh credentials plugin is unable to connect to slaves that have newer algorithms

      The keys from Jenkins (client) and slave (server below) have:

      fatal: no matching mac found:
      client: hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5
      server: hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com [preauth]
      

      Jenkins yields a trace:

      [06/22/15 14:49:05] [SSH] Opening SSH connection to 10.68.16.150:22.
      Key exchange was not finished, connection is closed.
      ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
      java.lang.IllegalStateException: Connection is not established!
      	at com.trilead.ssh2.Connection.getRemainingAuthMethods(Connection.java:1030)
      	at com.cloudbees.jenkins.plugins.sshcredentials.impl.TrileadSSHPublicKeyAuthenticator.getRemainingAuthMethods(TrileadSSHPublicKeyAuthenticator.java:88)
      	at com.cloudbees.jenkins.plugins.sshcredentials.impl.TrileadSSHPublicKeyAuthenticator.canAuthenticate(TrileadSSHPublicKeyAuthenticator.java:80)
      	at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.newInstance(SSHAuthenticator.java:207)
      	at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.newInstance(SSHAuthenticator.java:169)
      	at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1173)
      	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:701)
      	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:696)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      [06/22/15 14:49:06] Launch failed - cleaning up connection
      [06/22/15 14:49:06] [SSH] Connection closed.
      

      On our slaves we would like to have hmac-sha2-512 / hmac-sha2-256 but that is not supported by Trilead SSH.

        Attachments

          Issue Links

            Activity

            emma Emma Laurijssens created issue -
            Hide
            ollirajala Olli Rajala added a comment -

            This is a critical issue for us, because our security policy dictates that we must have quite strict ssh cipher settings configured to all our ssh servers.

            Show
            ollirajala Olli Rajala added a comment - This is a critical issue for us, because our security policy dictates that we must have quite strict ssh cipher settings configured to all our ssh servers.
            ollirajala Olli Rajala made changes -
            Field Original Value New Value
            Priority Minor [ 4 ] Major [ 3 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            I see the same issue when I connect the clean Ubuntu Server 14.04.4 LTS with the latest openssh-server from the update center

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited I see the same issue when I connect the clean Ubuntu Server 14.04.4 LTS with the latest openssh-server from the update center
            oleg_nenashev Oleg Nenashev made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-26495 [ JENKINS-26495 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            In my case I had to weaken the security settings and to use the common RSA algorithm

            Show
            oleg_nenashev Oleg Nenashev added a comment - In my case I had to weaken the security settings and to use the common RSA algorithm
            Hide
            biogerm Ryan An added a comment -

            I used another fork of Trilead ssh2 instead which has sha256 implemented.

            it's called ConnectBot sshlib. available on GitHub. https://github.com/connectbot/sshlib

            Show
            biogerm Ryan An added a comment - I used another fork of Trilead ssh2 instead which has sha256 implemented. it's called ConnectBot sshlib. available on GitHub. https://github.com/connectbot/sshlib
            hashar Antoine Musso made changes -
            Description The supported macs and kex methods in trilead are severely outdated, resulting in connection issues with properly secured ssh daemons on target machines. For instance:

            {{sshd[9800]: fatal: no matching mac found: client hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5 server hmac-sha2-256,hmac-sha2-512,umac-64@openssh.com,hmac-ripemd160 [preauth]
            }}

            In [JENKINS-14709|http://jenkins-ci.org/issue/14709] a suggestion is made to replace trilead with orion, but Orion is not being maintained either. Orion refers to Ganymed, but even that hasn't been looked at for almost 2 years: [Ganymed commits|https://code.google.com/archive/p/ganymed-ssh-2/source/default/commits]. It does seem to support hmac-sha2 macs though.
            The supported macs and kex methods in trilead are severely outdated, resulting in connection issues with properly secured ssh daemons on target machines. For instance:

            {noformat}
            sshd[9800]: fatal: no matching mac found: client hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5 server hmac-sha2-256,hmac-sha2-512,umac-64@openssh.com,hmac-ripemd160 [preauth]
            {noformat}

            In [JENKINS-14709|http://jenkins-ci.org/issue/14709] a suggestion is made to replace trilead with orion, but Orion is not being maintained either. Orion refers to Ganymed, but even that hasn't been looked at for almost 2 years: [Ganymed commits|https://code.google.com/archive/p/ganymed-ssh-2/source/default/commits]. It does seem to support hmac-sha2 macs though.
            hashar Antoine Musso made changes -
            Link This issue is duplicated by JENKINS-36873 [ JENKINS-36873 ]
            hashar Antoine Musso made changes -
            Description The supported macs and kex methods in trilead are severely outdated, resulting in connection issues with properly secured ssh daemons on target machines. For instance:

            {noformat}
            sshd[9800]: fatal: no matching mac found: client hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5 server hmac-sha2-256,hmac-sha2-512,umac-64@openssh.com,hmac-ripemd160 [preauth]
            {noformat}

            In [JENKINS-14709|http://jenkins-ci.org/issue/14709] a suggestion is made to replace trilead with orion, but Orion is not being maintained either. Orion refers to Ganymed, but even that hasn't been looked at for almost 2 years: [Ganymed commits|https://code.google.com/archive/p/ganymed-ssh-2/source/default/commits]. It does seem to support hmac-sha2 macs though.
            The supported macs and kex methods in trilead are severely outdated, resulting in connection issues with properly secured ssh daemons on target machines. For instance:

            {noformat}
            sshd[9800]: fatal: no matching mac found: client hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5 server hmac-sha2-256,hmac-sha2-512,umac-64@openssh.com,hmac-ripemd160 [preauth]
            {noformat}

            In [JENKINS-14709|http://jenkins-ci.org/issue/14709] a suggestion is made to replace trilead with orion, but Orion is not being maintained either. Orion refers to Ganymed, but even that hasn't been looked at for almost 2 years: [Ganymed commits|https://code.google.com/archive/p/ganymed-ssh-2/source/default/commits]. It does seem to support hmac-sha2 macs though.

            From JENKINS-36873 (dupe)

            The ssh credentials plugin is unable to connect to slaves that have newer algorithms

            The keys from Jenkins (client) and slave (server below) have:
            {noformat}
            fatal: no matching mac found:
            client: hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5
            server: hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com [preauth]
            {noformat}

            Jenkins yields a trace:
            {noformat}
            [06/22/15 14:49:05] [SSH] Opening SSH connection to 10.68.16.150:22.
            Key exchange was not finished, connection is closed.
            ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
            java.lang.IllegalStateException: Connection is not established!
            at com.trilead.ssh2.Connection.getRemainingAuthMethods(Connection.java:1030)
            at com.cloudbees.jenkins.plugins.sshcredentials.impl.TrileadSSHPublicKeyAuthenticator.getRemainingAuthMethods(TrileadSSHPublicKeyAuthenticator.java:88)
            at com.cloudbees.jenkins.plugins.sshcredentials.impl.TrileadSSHPublicKeyAuthenticator.canAuthenticate(TrileadSSHPublicKeyAuthenticator.java:80)
            at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.newInstance(SSHAuthenticator.java:207)
            at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.newInstance(SSHAuthenticator.java:169)
            at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1173)
            at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:701)
            at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:696)
            at java.util.concurrent.FutureTask.run(FutureTask.java:262)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at java.lang.Thread.run(Thread.java:745)
            [06/22/15 14:49:06] Launch failed - cleaning up connection
            [06/22/15 14:49:06] [SSH] Connection closed.
            {noformat}

            On our slaves we would like to have hmac-sha2-512 / hmac-sha2-256 but that is not supported by Trilead SSH.
            Hide
            hashar Antoine Musso added a comment -

            I have added a trace / some details from the duplicate task I have filled JENKINS-36873. As I understand it that Java installation is stall/no more updated by upstream and Jenkins core provides its own fork. Looks like the proper way to fix it would be to remove Trilead entirely and switch to another SSH implementation. Maybe Bouncy Castle?

            The workaround is to configure the slaves with some outdated algorithms supported by Trilead

            Our bug for my own reference https://phabricator.wikimedia.org/T103351

            Show
            hashar Antoine Musso added a comment - I have added a trace / some details from the duplicate task I have filled JENKINS-36873 . As I understand it that Java installation is stall/no more updated by upstream and Jenkins core provides its own fork. Looks like the proper way to fix it would be to remove Trilead entirely and switch to another SSH implementation. Maybe Bouncy Castle? The workaround is to configure the slaves with some outdated algorithms supported by Trilead Our bug for my own reference https://phabricator.wikimedia.org/T103351
            hashar Antoine Musso made changes -
            Component/s credentials-plugin [ 16523 ]
            Hide
            emma Emma Laurijssens added a comment -

            Couldn't find a similar issue when I created this one, but apparently it did exist.

            Show
            emma Emma Laurijssens added a comment - Couldn't find a similar issue when I created this one, but apparently it did exist.
            emma Emma Laurijssens made changes -
            Link This issue duplicates JENKINS-31549 [ JENKINS-31549 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 168847 ] JNJira + In-Review [ 183259 ]
            Hide
            jjmartinez Juan Martinez added a comment -

            Having this same issue in Jenkins 2.x and SSH plugin 1.11.

            Our problem is the key exchange when checking out SVN repos.

            Show
            jjmartinez Juan Martinez added a comment - Having this same issue in Jenkins 2.x and SSH plugin 1.11. Our problem is the key exchange when checking out SVN repos.
            Hide
            ygirouard Yanick Girouard added a comment -

            Has anyone found a working solution to this issue that doesn't involve changing accepted ciphers on the slaves?

            Show
            ygirouard Yanick Girouard added a comment - Has anyone found a working solution to this issue that doesn't involve changing accepted ciphers on the slaves?
            oleg_nenashev Oleg Nenashev made changes -
            Assignee Kohsuke Kawaguchi [ kohsuke ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            From what I see "no". Kohsuke was just a default assignee, but he rarely works on plugins now. Removed the assignee

            Show
            oleg_nenashev Oleg Nenashev added a comment - From what I see "no". Kohsuke was just a default assignee, but he rarely works on plugins now. Removed the assignee
            stephenconnolly Stephen Connolly made changes -
            Component/s credentials-plugin [ 16523 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-39805 [ JENKINS-39805 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-26379 [ JENKINS-26379 ]
            rsandell rsandell made changes -
            Assignee rsandell [ rsandell ]
            mc1arke Michael Clarke made changes -
            Assignee rsandell [ rsandell ] Michael Clarke [ mc1arke ]
            mc1arke Michael Clarke made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            mc1arke Michael Clarke made changes -
            Remote Link This issue links to "SHA256 and SHA512 HMAC Support (Web Link)" [ 15850 ]
            mc1arke Michael Clarke made changes -
            Remote Link This issue links to "Ed25519 Key Support (Web Link)" [ 15851 ]
            mc1arke Michael Clarke made changes -
            Remote Link This issue links to "ECDSA Key Support (Web Link)" [ 15852 ]
            Hide
            hashar Antoine Musso added a comment -

            A few various pull requests have been sent. Seems the active one is now https://github.com/jenkinsci/trilead-ssh2/pull/14 proposed by Michael Clarke (assignee of this task).

            Show
            hashar Antoine Musso added a comment - A few various pull requests have been sent. Seems the active one is now https://github.com/jenkinsci/trilead-ssh2/pull/14 proposed by Michael Clarke (assignee of this task).
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michael Clarke
            Path:
            src/com/trilead/ssh2/Connection.java
            src/com/trilead/ssh2/KnownHosts.java
            src/com/trilead/ssh2/crypto/CryptoWishList.java
            src/com/trilead/ssh2/crypto/dh/DhExchange.java
            src/com/trilead/ssh2/crypto/dh/DhGroupExchange.java
            src/com/trilead/ssh2/crypto/digest/HMAC.java
            src/com/trilead/ssh2/crypto/digest/HashForSSH2Types.java
            src/com/trilead/ssh2/crypto/digest/JreMessageDigestWrapper.java
            src/com/trilead/ssh2/crypto/digest/MAC.java
            src/com/trilead/ssh2/crypto/digest/MD5.java
            src/com/trilead/ssh2/crypto/digest/MessageMac.java
            src/com/trilead/ssh2/crypto/digest/SHA1.java
            src/com/trilead/ssh2/transport/KexManager.java
            src/com/trilead/ssh2/transport/KexState.java
            http://jenkins-ci.org/commit/trilead-ssh2/3aaec8394cb949499061186219ab9c513c0d9eea
            Log:
            Merge pull request #14 from jenkinsci/SHA256-and-SHA512-HMAC-support

            JENKINS-33021 Add support for SHA256 and SHA512 HMAC algorithms

            Compare: https://github.com/jenkinsci/trilead-ssh2/compare/d0178c21e393...3aaec8394cb9

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michael Clarke Path: src/com/trilead/ssh2/Connection.java src/com/trilead/ssh2/KnownHosts.java src/com/trilead/ssh2/crypto/CryptoWishList.java src/com/trilead/ssh2/crypto/dh/DhExchange.java src/com/trilead/ssh2/crypto/dh/DhGroupExchange.java src/com/trilead/ssh2/crypto/digest/HMAC.java src/com/trilead/ssh2/crypto/digest/HashForSSH2Types.java src/com/trilead/ssh2/crypto/digest/JreMessageDigestWrapper.java src/com/trilead/ssh2/crypto/digest/MAC.java src/com/trilead/ssh2/crypto/digest/MD5.java src/com/trilead/ssh2/crypto/digest/MessageMac.java src/com/trilead/ssh2/crypto/digest/SHA1.java src/com/trilead/ssh2/transport/KexManager.java src/com/trilead/ssh2/transport/KexState.java http://jenkins-ci.org/commit/trilead-ssh2/3aaec8394cb949499061186219ab9c513c0d9eea Log: Merge pull request #14 from jenkinsci/SHA256-and-SHA512-HMAC-support JENKINS-33021 Add support for SHA256 and SHA512 HMAC algorithms Compare: https://github.com/jenkinsci/trilead-ssh2/compare/d0178c21e393...3aaec8394cb9
            Hide
            paladox paladox added a comment -

            Resolving the issue as a new trilead-ssh2 release has been done with ^^ change. All that needs doing is jenkins to be updated now.

            Show
            paladox paladox added a comment - Resolving the issue as a new trilead-ssh2 release has been done with ^^ change. All that needs doing is jenkins to be updated now.
            paladox paladox made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            Hide
            paladox paladox added a comment -

            Re opening as still in progress.

            Show
            paladox paladox added a comment - Re opening as still in progress.
            paladox paladox made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Hide
            markl_lagendijk Mark Lagendijk added a comment -
            Show
            markl_lagendijk Mark Lagendijk added a comment - You can follow progress here: https://issues.jenkins-ci.org/browse/JENKINS-43610
            Hide
            markl_lagendijk Mark Lagendijk added a comment - - edited

            Edit: removed message about failure to patch Jenkins with the new trilead library. Building Jenkins from source with the new trilead library did work.

            Show
            markl_lagendijk Mark Lagendijk added a comment - - edited Edit: removed message about failure to patch Jenkins with the new trilead library. Building Jenkins from source with the new trilead library did work.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Michael Clarke
            Path:
            core/pom.xml
            http://jenkins-ci.org/commit/jenkins/b17d0763709be35d39f16d6af7afaf765ac6cf92
            Log:
            Bump Trilead version to receive a number of security enhancements
            JENKINS-41606JENKINS-33021JENKINS-26379JENKINS-31549

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michael Clarke Path: core/pom.xml http://jenkins-ci.org/commit/jenkins/b17d0763709be35d39f16d6af7afaf765ac6cf92 Log: Bump Trilead version to receive a number of security enhancements JENKINS-41606 JENKINS-33021 JENKINS-26379 JENKINS-31549
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            It has been merged towards 2.58

            Not a backporting candidate

            Show
            oleg_nenashev Oleg Nenashev added a comment - It has been merged towards 2.58 Not a backporting candidate
            Hide
            danielbeck Daniel Beck added a comment -

            In 2.58.

            Show
            danielbeck Daniel Beck added a comment - In 2.58.
            danielbeck Daniel Beck made changes -
            Resolution Fixed [ 1 ]
            Status Reopened [ 4 ] Resolved [ 5 ]
            nperrenoud Nicolas Perrenoud made changes -
            Link This issue is related to JENKINS-44046 [ JENKINS-44046 ]
            Hide
            hashar Antoine Musso added a comment -

            Thank you Michael Clarke ! That works perfectly and we use:

             

            server: hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com [preauth]
            
            Show
            hashar Antoine Musso added a comment - Thank you Michael Clarke ! That works perfectly and we use:   server: hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com [preauth]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal OSS-868 (Web Link)" [ 18814 ]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal CLTS-1375 (Web Link)" [ 19240 ]

              People

              Assignee:
              mc1arke Michael Clarke
              Reporter:
              emma Emma Laurijssens
              Votes:
              13 Vote for this issue
              Watchers:
              20 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: