-
Bug
-
Resolution: Fixed
-
Critical
-
Versions:
Jenkins: 2.51
SSH Agent Plugin: 1.14
SSH Credentials plugin: 1.13
SSH Slaves Plugin : 1.15
Ubuntu 14.04, 16.04
-
Powered by SuggestiMate
SSH agent isn't launched after the latest update, complaining about missing records in the known hosts file.
Nevertheless, the records do exist. I've tried to connect manually with ssh, everything was fine, ssh reports that it has found the host in known_hosts.
Probable reason is the hashed host name in known_hosts.
Here is the log from the Jenkins slave launch page:
[03/21/17 11:40:34] [SSH] Opening SSH connection to xxx.xxx.xxx.xxx:22 [03/21/17 11:40:34] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to xxx.xxx.xxx.xxx:22 at com.trilead.ssh2.Connection.connect(Connection.java:818) at com.trilead.ssh2.Connection.connect(Connection.java:687) at com.trilead.ssh2.Connection.connect(Connection.java:601) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:93) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:230) at com.trilead.ssh2.Connection.connect(Connection.java:770) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) ... 1 more [03/21/17 11:40:34] Launch failed - cleaning up connection [03/21/17 11:40:34] [SSH] Connection closed.
And this is the debug output from ssh, showing that it has found a record in the second line of known_hosts.
$ sudo -u jenkins -g jenkins ssh -v jenkins@xxx.xxx.xxx.xxx OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: Applying options for * debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22. debug1: Connection established. debug1: identity file /var/lib/jenkins/.ssh/id_rsa type 1 ... debug1: sending SSH2_MSG_KEX_ECDH_INIT debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key: ECDSA xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx debug1: Host 'xxx.xxx.xxx.xxx.xxx' is known and matches the ECDSA host key. debug1: Found key in /var/lib/jenkins/.ssh/known_hosts:2 debug1: ssh_ecdsa_verify: signature correct ... Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-67-generic x86_64)
Additional logs from Jenkins system log:
Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager kex_algo=diffie-hellman-group14-sha1 Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager server_host_key_algo=ssh-rsa Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager enc_algo_client_to_server=aes256-ctr Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager enc_algo_server_to_client=aes256-ctr Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager mac_algo_client_to_server=hmac-sha1 Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager mac_algo_server_to_client=hmac-sha1 Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager comp_algo_client_to_server=none Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.KexManager comp_algo_server_to_client=none Mar 21, 2017 12:06:08 PM FINE com.trilead.ssh2.transport.TransportManager Receive thread: error in receiveLoop java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:745) Mar 21, 2017 12:06:08 PM FINER com.trilead.ssh2.transport.TransportManager Receive thread: back from receiveLoop Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager kex_algo=diffie-hellman-group14-sha1 Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager server_host_key_algo=ssh-rsa Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager enc_algo_client_to_server=aes256-ctr Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager enc_algo_server_to_client=aes256-ctr Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager mac_algo_client_to_server=hmac-sha1 Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager mac_algo_server_to_client=hmac-sha1 Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager comp_algo_client_to_server=none Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.KexManager comp_algo_server_to_client=none Mar 21, 2017 12:06:10 PM FINE com.trilead.ssh2.transport.TransportManager Receive thread: error in receiveLoop java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:777) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:745) Mar 21, 2017 12:06:10 PM FINER com.trilead.ssh2.transport.TransportManager Receive thread: back from receiveLoop
File /var/lib/jenkins/.ssh/known_hosts contains strings, looking like being base64-encoded, delimited by '|'. Here is the sample.
|1|DAg ... o... 1ll9wI=| ... .... tIrM= ecdsa-sha2-nistp256 xxxxxx..... bmlzdHAyNTYAAAAIbm................. .................xxxxxxxxxxxoKEHF3Vr0q685jI2+6vWjvAAG4lz5Ckujy9k=
- is blocked by
-
JENKINS-44832 SSH Slaves plugin fails the host verification of slaves with an IllegalArgumentException
-
- Resolved
-
- is duplicated by
-
JENKINS-44803 Master 2.60.1-rc1 (2.58-2.64) ssh agents won't connect with "Manually provided key Verification Strategy"
-
- Closed
-
- relates to
-
JENKINS-44830 NullPointerException when upgrading to ssh-slaves 1.18
-
- Resolved
-
- links to
[JENKINS-42959] Failed known_hosts verification for SSH agent
srl295's fix solved it for me as well. I purged the other lines from ~/.ssh/known_hosts just in case.
I logged in to the Master server in a command line shell and sudo'd (sudo -su <user>) to the user running the Jenkins war file. I ran the ssh command to connect to each of the Jenkins Agent machines and had no issue connecting. I did this with both the machine name and the fully qualified domain name (e.g. machine1, and machine1.company-name.com). From what I could see the ssh ~/.ssh/known_hosts file contained valid information on the Agent machines and had correct permissions of 644.
On the Jenkins Master web page, I went to the Nodes, Agent, Configure page (http://<master>:8080/computer/<agent>/) and switched from the "Known hosts file Verification Strategy" to the "Manually trusted key Verification Strategy" then brought the Agents online with no issues. I then switched the Agent configuration back to the "Known hosts file Verification Strategy", took the Agents offline and brought them back online again with no issues.
I looked at the time stamp on the ~/.ssh/known_hosts file and verified its contents hadn't changed. Its almost as if the known_hosts file being checked isn't the file for the user id executing the Jenkins war file. Either way, this is an issue that needs to be addressed sooner rather than later.
I'll try to summarize what we already know here from reading the original post, and the first few comments.
- Jenkins uses a Java ssh library. It does not use the same ssh as is used on the command line. The exception found in the original issue description makes this clear
The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:591)
2. As stated by srl295 above:
I needed this workaround from the mailing list:
> ssh -o HostKeyAlgorithms=ssh-rsa slave2.example.com
The implication below is that Jenkins is using weaker encryption.
3. The workaround (not fix) seems to be to switch the node over to "Manually trusted key Verification Strategy".
The actual fix, then, seems to be for the com.trilead.ssh2 library to be updated to handle advances in which kinds of ssh keys are considered secure at this point. But a search on the internet makes it clear it is no longer being worked on by the original developers, though someone appears to be maintaining it somewhat here: https://github.com/jenkinsci/trilead-ssh2
If that Java library is not going to be updated, then perhaps there needs to be a push to find another Java library for managing ssh connections.
sshj seems to be the best option at the moment: https://github.com/hierynomus/sshj every thing else I am finding seems pretty dormant.
Apache Mina SSHD might also be an option: http://mina.apache.org/sshd-project/
Code changed in jenkins
User: Michael Clarke
Path:
src/com/trilead/ssh2/KnownHosts.java
test/com/trilead/ssh2/KnownHostsTest.java
http://jenkins-ci.org/commit/trilead-ssh2/7733cf2fba6f18126c754c409689731292ff40cb
Log:
JENKINS-42959 Return host algorithms if they're in known hosts and we support them
When the fixed ssh-slave-plugin will available as an update ? Our jenkins build setup is broken due to this
ayappan the fix linked above isn't a complete fix for this issue. I currently have a number of pull requests raised against the Jenkins fork of Trilead to enhance Jenkins's SSH client capabilities. Even once these have been reviewed and merged, we still need to plan how we release this as Trilead is bundled in Jenkins core rather than with a plugin, so upgrading the version bundled in the core won't resolve the issue for everyone until that also gets into an LTS release and people have upgraded to it.
Is it sensible to rollback the plugin to an earlier release until this problem is fixed?
Is it sensible to rollback the plugin to an earlier release until this problem is fixed?
No, definitely not. There's no requirement to use the known hosts verification strategy- you can use the automatic trusting strategy that purely checks the host key doesn't change, and (optionally) allows an admin to approve the first key that's presented. Rolling back the plugin would expose users to the potential of a Man-In-The-Middle attack,
Thank you. Changing to 'Non verifying verification strategy' enabled the slave to connect again.
Code changed in jenkins
User: Michael Clarke
Path:
pom.xml
src/com/trilead/ssh2/Connection.java
src/com/trilead/ssh2/ConnectionInfo.java
src/com/trilead/ssh2/KnownHosts.java
src/com/trilead/ssh2/ServerHostKeyVerifier.java
src/com/trilead/ssh2/auth/AuthenticationManager.java
src/com/trilead/ssh2/crypto/CertificateDecoder.java
src/com/trilead/ssh2/crypto/PEMDecoder.java
src/com/trilead/ssh2/crypto/PEMStructure.java
src/com/trilead/ssh2/signature/DSAKeyAlgorithm.java
src/com/trilead/ssh2/signature/DSAPrivateKey.java
src/com/trilead/ssh2/signature/DSAPublicKey.java
src/com/trilead/ssh2/signature/DSASHA1Verify.java
src/com/trilead/ssh2/signature/DSASignature.java
src/com/trilead/ssh2/signature/ED25519KeyAlgorithm.java
src/com/trilead/ssh2/signature/KeyAlgorithm.java
src/com/trilead/ssh2/signature/KeyAlgorithmManager.java
src/com/trilead/ssh2/signature/RSAKeyAlgorithm.java
src/com/trilead/ssh2/signature/RSAPrivateKey.java
src/com/trilead/ssh2/signature/RSAPublicKey.java
src/com/trilead/ssh2/signature/RSASHA1Verify.java
src/com/trilead/ssh2/signature/RSASignature.java
src/com/trilead/ssh2/transport/KexManager.java
test/com/trilead/ssh2/signature/DSAKeyAlgorithmTest.java
test/com/trilead/ssh2/signature/ED25519KeyAlgorithmTest.java
test/com/trilead/ssh2/signature/RSAKeyAlgorithmTest.java
test/com/trilead/ssh2/signature/ed25519-testkey-protected.txt
test/com/trilead/ssh2/signature/ed25519-testkey-unprotected.txt
http://jenkins-ci.org/commit/trilead-ssh2/b3ca9db0df6652c655e6c472bc7a15dc52f33b76
Log:
Merge pull request #13 from jenkinsci/JENKINS-42959-support-ED25519-keys
Support Ed25519 asymmetric keys
Compare: https://github.com/jenkinsci/trilead-ssh2/compare/3a9205ecd330...b3ca9db0df66
Code changed in jenkins
User: Michael Clarke
Path:
src/com/trilead/ssh2/crypto/SimpleDERReader.java
src/com/trilead/ssh2/signature/ECDSAKeyAlgorithm.java
src/com/trilead/ssh2/signature/KeyAlgorithmManager.java
test/com/trilead/ssh2/signature/ECDSAKeyAlgorithmTest.java
http://jenkins-ci.org/commit/trilead-ssh2/d0178c21e3939fa057997eb98987761b1b912e97
Log:
Merge pull request #12 from jenkinsci/JENKINS-42959-support-ECDSA-keys
Support ECDSA asymmetric keys
Compare: https://github.com/jenkinsci/trilead-ssh2/compare/b3ca9db0df66...d0178c21e393
Code changed in jenkins
User: Michael Clarke
Path:
src/com/trilead/ssh2/KnownHosts.java
test/com/trilead/ssh2/KnownHostsTest.java
http://jenkins-ci.org/commit/trilead-ssh2/63d9a981eec88c46b8b73ce54f8980f40c6b1927
Log:
JENKINS-42959 Return host algorithms if they're in known hosts and we support them
Code changed in jenkins
User: Michael Clarke
Path:
src/com/trilead/ssh2/KnownHosts.java
test/com/trilead/ssh2/KnownHostsTest.java
http://jenkins-ci.org/commit/trilead-ssh2/4e0e9e110adb1101bd9c8f0712e0d427cb020349
Log:
JENKINS-42959 Return host algorithms if in known_host and we support them (#11)
JENKINS-42959 Return host algorithms if they're in known hosts and we support them
- Update test following latest merges
This has been resolved in Jenkins 2.58 with an update to Trilead to support ECDSA and ED25519 keys. This does not change the messages referenced in one of the above comments since manually entered keys still only support DSA and RSA algorithms, however a separate change will be raised against ssh-slaves-plugin to resolve this in the near future.
Jenkins: 2.58
SSH Slaves plugin: 1.17
Connection to solaris (sparc or x86) hosts is not possible.
Entry in ~/.ssh/known_hosts is present and uses ssh-rsa algorithm. Host is configured with “Known hosts file Verification Strategy”.
Log File contains following entries:
[05/02/17 09:28:18] [SSH] Opening SSH connection to quark:22. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to quark:22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: Cannot read full block, EOF reached. at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:81) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/02/17 09:28:18] Launch failed - cleaning up connection [05/02/17 09:28:18] [SSH] Connection closed.
Another host (Linux) with ecdsa-sha2-nistp256 works.
dmazuronak your error seems to be different - Trilead doesn't seem to be getting enough data during the key exchange phase to allow decryption to work properly. This is unrelated to this original issue, so could you open a new issue please, and provide as much detail about your SSH host's software versions and configuration in it as possible please?
Since latest Jenkins update, the ssh connection to our Debian slave has been lost. We were successfully using "Manual trusted key verification strategy" before. Now on launching the slave, I get these results:
"Manually trusted key verification strategy"
[05/02/17 09:55:20] [SSH] WARNING: The SSH key for this host is not currently trusted. Connections will be denied until this new key is authorised. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to *** at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:548) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/02/17 09:55:20] Launch failed - cleaning up connection [05/02/17 09:55:20] [SSH] Connection closed.
"Known hosts file strategy"
[05/02/17 09:59:42] [SSH] Opening SSH connection to *** Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to *** at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback. at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:543) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more Caused by: java.lang.IllegalArgumentException: Unknown hostkey type ecdsa-sha2-nistp256 at com.trilead.ssh2.KnownHosts.decodeHostKey(KnownHosts.java:607) at com.trilead.ssh2.KnownHosts.verifyHostkey(KnownHosts.java:569) at hudson.plugins.sshslaves.verifiers.KnownHostsFileKeyVerificationStrategy.verify(KnownHostsFileKeyVerificationStrategy.java:62) at hudson.plugins.sshslaves.SSHLauncher$3.verifyServerHostKey(SSHLauncher.java:1279) at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:539) ... 3 more [05/02/17 09:59:42] Launch failed - cleaning up connection [05/02/17 09:59:42] [SSH] Connection closed.
I understand the last part, that's due to the different key-types. However, the recent updates on Jenkins seem to have broken the original workaround of using the "manually trusted" strategy.
pjaytycy I suspect the "manually trusted" Known Hosts strategy failure is due to the server now responding with ECDSA or ED25519 keys (added to the Trilead supported list in the latest release) which means it no longer matches the RSA or DSA key that the server would have previously responded with and had been saved in your trusted key list.
This should be fixable in ssh-slaves without a new Trilead release, although dmazuronak report will require a Trilead update. Are you able to confirm what JVM you're running Jenkins master on, as I'm surprised to see it reporting "Unknown hostkey type ecdsa-sha2-nistp256"?
Just upgraded to Jenkins 2.58, SSH Slaves plugin 1.17.
Agent launch fails:
"Known hosts file"
[05/03/17 00:05:32] [SSH] Opening SSH connection to X.X.X.X:22. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to X.X.X.X:22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: Cannot read full block, EOF reached. at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:81) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/03/17 00:05:32] Launch failed - cleaning up connection [05/03/17 00:05:32] [SSH] Connection closed.
"Non verifying"
[05/03/17 00:02:35] [SSH] Opening SSH connection to X.X.X.X:22. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to X.X.X.X:22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: Cannot read full block, EOF reached. at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:81) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/03/17 00:02:35] Launch failed - cleaning up connection [05/03/17 00:02:35] [SSH] Connection closed.
mc1arke, this is the JVM info from Jenkins master:
java.runtime.name | OpenJDK Runtime Environment |
java.runtime.version | 1.8.0_121-8u121-b13-1~bpo8+1-b13 |
This is on Debian 8.7, using openjdk from the jessie-backports packages.
mc1arke I see this in my known_host file
1 | mr96ncUVquOccX45cV5eJgJmMto= | fTz8Qek7t7rHTTUwNd4jlZzuA8c= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGs74V0LaOXLN2zqa+wZgSziUrBmc5nTN1X+EBqIVY6v2LMyxXBXJ4k+BKm5n51lLYGhr+iygVVVMRYXWeiNqtg= |
1 | 307Mf25a48jG7A8k9X7p1mxUXko= | m29JXn9wlHmhGwL6/7CUAMOaAP4= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGs74V0LaOXLN2zqa+wZgSziUrBmc5nTN1X+EBqIVY6v2LMyxXBXJ4k+BKm5n51lLYGhr+iygVVVMRYXWeiNqtg= |
1 | sQj3Qv4SAg6XQsZ358nyEg+k+Go= | reo+oJiMg8kIRzlFQS6IMi09AwA= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCSyFsBAw7UBZCSKyrealtJjGcHM2cv/yNKVRX1USMAo7Ofer2nF/HbM3gCp08Yr3xEs3vAGDuTAxnwmW+7lWlo= |
1 | bzyQVpCG/2mex0UbAfgWqhaniys= | Wvv0kcMnj0ydXQQ6jJcYhx2Ce7Y= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCSyFsBAw7UBZCSKyrealtJjGcHM2cv/yNKVRX1USMAo7Ofer2nF/HbM3gCp08Yr3xEs3vAGDuTAxnwmW+7lWlo= |
1 | j3X0MtD5xu0bzCzClkRca7dssa8= | qap2zB/7hUC/mrJ/0ZVgt/wKq4s= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ== |
But i do get this
[05/03/17 12:58:51] [SSH] Opening SSH connection to jenkins-slave-01:22. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to jenkins-slave-01:22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback. at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:543) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more Caused by: java.lang.IllegalArgumentException: Unknown hostkey type ssh-ed25519 at com.trilead.ssh2.KnownHosts.decodeHostKey(KnownHosts.java:607) at com.trilead.ssh2.KnownHosts.verifyHostkey(KnownHosts.java:569) at hudson.plugins.sshslaves.verifiers.KnownHostsFileKeyVerificationStrategy.verify(KnownHostsFileKeyVerificationStrategy.java:62) at hudson.plugins.sshslaves.SSHLauncher$3.verifyServerHostKey(SSHLauncher.java:1279) at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:539) ... 3 more [05/03/17 12:58:51] Launch failed - cleaning up connection [05/03/17 12:58:51] [SSH] Connection closed.
when i have it set to look at known_host
this https://github.com/jenkinsci/ssh-slaves-plugin/blob/11ec7e7a10ae97ba7bef02dcc3a353136ccc576d/src/main/java/hudson/plugins/sshslaves/verifiers/ManuallyProvidedKeyVerificationStrategy.java#L79 line looks like it needs updated to support newer macs.
Code changed in jenkins
User: Michael Clarke
Path:
src/com/trilead/ssh2/KnownHosts.java
test/com/trilead/ssh2/KnownHostsTest.java
http://jenkins-ci.org/commit/trilead-ssh2/25a98a6db3a939f3128439082c3541522a068aef
Log:
JENKINS-42959 Correctly check host key algorithm
Code changed in jenkins
User: Michael Clarke
Path:
src/com/trilead/ssh2/KnownHosts.java
test/com/trilead/ssh2/KnownHostsTest.java
http://jenkins-ci.org/commit/trilead-ssh2/2f01ab94bfc72edab8c06554b7139e733c2cd540
Log:
Merge pull request #21 from mc1arke/jenkins-42959-key-format-mismatch
JENKINS-42959 Correctly check host key algorithm
Compare: https://github.com/jenkinsci/trilead-ssh2/compare/63b2b8505466...2f01ab94bfc7
Code changed in jenkins
User: Michael Clarke
Path:
core/pom.xml
http://jenkins-ci.org/commit/jenkins/f36917c210d52ce7fc1f4bca71c9f1baf11c4647
Log:
[JENKINS-42959, JENKINS-44046, JENKINS-43979] - Bump Trilead to build217-jenkins-10 (#2872)
JENKINS-42959 Correctly compare key algorithms during key verification
[FIXED JENKINS-44046][FIXED JENKINS-43979] Use a larger default key size to fix issues when using SHA256 MACs during Diffe-Helman key exchange against older versions of OpenSSH
Fixes issues encountered following the recent move to the latest Trilead version. These fix 2 underlying issues:
diffe-helman-sha256 Key Exchange needs at least 2048 bits of data to generate a key against OpenSSH 6.4 and below.
Known hosts comparison incorrectly compares the hostname against the key value meaning key verification always fails when using known hosts files
Upgraded from 2.54 to 2.58 today and all of my slaves failed to connect with:
[05/05/17 16:52:45] [SSH] Opening SSH connection to 1.2.3.4:22. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to 1.2.3.4:22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: Cannot read full block, EOF reached. at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:81) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/05/17 16:52:45] Launch failed - cleaning up connection [05/05/17 16:52:45] [SSH] Connection closed.
I had to downgrade back to 2.54 again.
I submitted https://github.com/jenkinsci/ssh-slaves-plugin/pull/51 as a starting point for fixing this in `ssh-slaves-plugin`.
mc1arke Is this fixed by the trilead update in 2.59, or are there followup changes needed? This is assigned to ssh-slaves, so I guess yes?
danielbeck there are 2 scenarios that fail in the above reports, one if fully fixed by the Trilead update, another needs a further update to SSH slaves. I'll endeavor to get a pull request raised against SSH slaves at some point today to try and close down the SSH connection issues.
I still have this problem in 2.59; downgrading to 2.57 fixes the problem.
That's attempting to start an agent on an oldish Mac Mini.
The version 2.59 solved problems for Solaris hosts. Our SLES (Linux) server are still offline. There is the entries from the log file:
[05/09/17 19:33:43] [SSH] Opening SSH connection to <HOST>:22. [05/09/17 19:33:43] [SSH] The SSH key presented by the remote host does not match the key saved in the Known Hosts file against this host. Connections to this host will be denied until the two keys match. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to bllttestappv008:22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:548) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/09/17 19:33:43] Launch failed - cleaning up connection [05/09/17 19:33:43] [SSH] Connection closed.
The host is configured to use "Known host file verification strategy" . And the file ~.ssh/known_hosts allready containts entry for the host
<HOST>,<IPv4> ecdsa-sha2-nistp256 AAAA...
The "Manually trusted key verification strategy" works well.
I had this problem with the new ssh keys. I forgot how I fixed it but I remember @m1clarke gave me a command to run that fixed it for me.
Hi,
I have just faced the same issue. Tried ssh-keyscan -H $MYHOST >> /var/lib/jenkins/.ssh/known_hosts 2>&1 and by ssh and copying the generated hash on $user/.ssh/known_hosts, without success.
Not sure if my update is helpful, but just mentioning that this used to work straightforward before installing the latest version, maybe something specific to my setup.
Thanks in advance.
[05/11/17 01:13:03] [SSH] Opening SSH connection to :22. [05/11/17 01:13:03] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file. Key exchange was not finished, connection is closed. java.io.IOException: There was a problem while connecting to :22 at com.trilead.ssh2.Connection.connect(Connection.java:834) at com.trilead.ssh2.Connection.connect(Connection.java:703) at com.trilead.ssh2.Connection.connect(Connection.java:617) at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1265) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:790) at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:785) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:786) ... 9 more Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:548) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more [05/11/17 01:13:03] Launch failed - cleaning up connection [05/11/17 01:13:03] [SSH] Connection closed.
@Jose Roberto Almaraz da Cunha Junior Hi, try changing to the Jenkins user then try ssh into the host (it dosent matter if you doint use the correct key) it will then Store it in known_host.
sudo su jenkins
ssh <host>
The command ssh-keyscan get 2 type of keys for my problem host: ssh-rsa and ecdsa-sha2-nistp256. OS SSH client used ecdsa-sha2-nistp256, which is also stored in known_hosts file, and successful established the connection. I thing that SSH Slave tried to check only ssh-rsa key and failed.
@paladox, thanks! I was wondering if its maybe a java version mismatch (Jenkins server using 8 and node using 7). If I change to jenkins user, nothing happens. Thought of pasting an strace of the su, but it would be too much info
[root@jenkins-sv jenkins]# su - jenkins
[root@jenkins-sv jenkins]# whoami
root
[root@jenkins-sv jenkins]# cat /etc/passwd | grep jenkins
jenkins:x:498:498:Jenkins Automation Server:/var/lib/jenkins:/bin/false
[root@jenkins-sv jenkins]# cat /var/lib/jenkins/.ssh/known_hosts
node01,ip ssh-rsa HASH==
node02,ip ssh-rsa HASH==
node03,ip ssh-rsa HASH==
node04,ip ssh-rsa HASH==
[root@jenkins-sv jenkins]# id jenkins
uid=498(jenkins) gid=498(jenkins) groups=498(jenkins)
============= Jenkins Node Launch
[05/18/17 14:41:36] [SSH] Checking java version of java
[05/18/17 14:41:36] [SSH] java -version returned 1.7.0_101.
[05/18/17 14:41:36] [SSH] Starting sftp client.
[05/18/17 14:41:36] [SSH] Copying latest slave.jar...
[05/18/17 14:41:36] [SSH] Copied 719,269 bytes.
Expanded the channel window size to 4MB
[05/18/17 14:41:36] [SSH] Starting slave process: cd "/path" && java -jar slave.jar
<===[JENKINS REMOTING CAPACITY]===>channel started
Slave JVM has not reported exit code. Is it still running?
[05/18/17 14:41:40] Launch failed - cleaning up connection
[05/18/17 14:41:40] [SSH] Connection closed.
ERROR: Connection terminated
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
Caused: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
jralmaraz Your issue isn't related to the SSH verification - it's the Slave process failing to start once the SSH connection has started. If you are running Java 7 on the slave machine then this will be the cause of your problem: both Slave and Master need to be on Java 8.
Code changed in jenkins
User: Michael Clarke
Path:
pom.xml
src/main/java/hudson/plugins/sshslaves/SSHLauncher.java
src/main/java/hudson/plugins/sshslaves/verifiers/JenkinsTrilead9VersionSupport.java
src/main/java/hudson/plugins/sshslaves/verifiers/KnownHostsFileKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/ManuallyProvidedKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/ManuallyTrustedKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/SshHostKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/TrileadVersionSupportManager.java
src/test/java/hudson/plugins/sshslaves/verifiers/ManuallyProvidedKeyVerificationStrategyTest.java
src/test/java/hudson/plugins/sshslaves/verifiers/TrileadVersionSupportManagerTest.java
http://jenkins-ci.org/commit/ssh-slaves-plugin/9a0fdc706d2862d79b8bc1c4c63df87049f4a71c
Log:
[FIXED JENKINS-42959] Specify preferred host keys during connect
Code changed in jenkins
User: Stephen Connolly
Path:
pom.xml
src/main/java/hudson/plugins/sshslaves/SSHLauncher.java
src/main/java/hudson/plugins/sshslaves/verifiers/JenkinsTrilead9VersionSupport.java
src/main/java/hudson/plugins/sshslaves/verifiers/KeyParseException.java
src/main/java/hudson/plugins/sshslaves/verifiers/KnownHostsFileKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/ManuallyProvidedKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/ManuallyTrustedKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/SshHostKeyVerificationStrategy.java
src/main/java/hudson/plugins/sshslaves/verifiers/TrileadVersionSupportManager.java
src/test/java/hudson/plugins/sshslaves/verifiers/ManuallyProvidedKeyVerificationStrategyTest.java
src/test/java/hudson/plugins/sshslaves/verifiers/TrileadVersionSupportManagerTest.java
http://jenkins-ci.org/commit/ssh-slaves-plugin/67bbbffac69974b719c3177a1f0036126f227c52
Log:
Merge pull request #54 from mc1arke/jenkins-42959-specify-hostkey-algorithms
[FIXED JENKINS-42959] Specify preferred host keys during connect
Compare: https://github.com/jenkinsci/ssh-slaves-plugin/compare/860a3a92b4cc...67bbbffac699
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/hudson/plugins/sshslaves/SSHLauncher.java
src/test/java/hudson/plugins/sshslaves/SSHLauncherTest.java
http://jenkins-ci.org/commit/ssh-slaves-plugin/7487db55f6b6b5adadbbc6884fad56113772d540
Log:
[FIXED JENKINS-44830] Correct NPE after upgrading caused by fix of JENKINS-42959.
Code changed in jenkins
User: Stephen Connolly
Path:
pom.xml
src/main/java/hudson/plugins/sshslaves/SSHLauncher.java
src/test/java/hudson/plugins/sshslaves/SSHLauncherTest.java
http://jenkins-ci.org/commit/ssh-slaves-plugin/367bfdfa1d297f88a840e429b20acba0a34e0032
Log:
Merge pull request #61 from jglick/NPE-JENKINS-44830
JENKINS-44830 NPE after upgrading caused by fix of JENKINS-42959
Compare: https://github.com/jenkinsci/ssh-slaves-plugin/compare/ce18ed6aa6b4...367bfdfa1d29
It's fixed for me now - tested on Jenkins 2.66, SSH Slaves plugin 1.20
The problem seems to be back for me Jenkins 2.69, SSH Slaves plugin 1.20
Having just upgraded from 2.68 to 2.69, with SSH Slaves plugin 1.20, it's still WORKING for me.
olenads Please provide more information, such as the exact error message (with stack trace), as well as what the known_hosts entry looks like.
I have these versions:
Jenkins 2.60.2
SSH Slaves Plugin 1.20
and am still getting this error:
Warning: no key algorithms provided; JENKINS-42959 disabled [08/14/17 18:47:10] [SSH] Opening SSH connection to xxxxxx:22. [08/14/17 18:47:10] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file.
This message means exactly what it says.
No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file
In your agent configuration, for "Host Key Verification Strategy" you probably have "Known hosts file verification strategy". The help text for that says:
Checks the known_hosts file (~/.ssh/known_hosts) for the user Jenkins is executing under, to see if an entry exists that matches the current connection.
This method does not make any updates to the Known Hosts file, instead using the file as a read-only source and expecting someone with suitable access to the appropriate user account on the Jenkins master to update the file as required, potentially using the ssh hostname command to initiate a connection and update the file appropriately.
So that's what you need to do. If you need more help, ask on the mailing list, not here, since it's not a bug.
Yes, and I've been asking our elusive Sys Admin to please ssh from one host to the other in order to populate the known_hosts. This never used to be required by the plugin in the past (or if it was, I was not aware), but I guess it is now.
Meanwhile, I noticed there is a dropdown menu under Node configuration, and I selected "Manually provided key Verification Strategy", and then I pasted the contents of id_rsa.pub into the "SSH Key" textbox, but got this error:
[08/17/17 10:48:58] [SSH] WARNING: The SSH key for this host does not match the key required in the connection configuration. Connections will be denied until until the host key matches the configuration key. Key exchange was not finished, connection is closed.
I then tried "Non verifying Verification Strategy", and got this error:
[08/17/17 10:46:28] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection. ERROR: Server rejected the 1 private key(s) for jenkins (credentialId:e85865a0-e6eb-4de3-a77e-b0fbbc9ec8c4/method:publickey) [08/17/17 10:46:28] [SSH] Authentication failed. Authentication failed.
So it seems like the "Manually provided key" method will work for me, but there is an issue with key mismatch. So, I'll follow up with our sys admin.
mcsf Which id_rsa.pub was it you copied in to the configuration? Your personal one (~/.ssh/id_rsa.pub), or the one for the host's public key (/etc/sshd/id_rsa.pub or similar)? It needs to be the latter for this to work.
Your final failure indicates that your private key (used for authentication) does not match any public key saved on the host, which no amount of key verification is going to fix.
Using Non verifying Verification Strategy is strongly discouraged, unless you can verify the identity of the host through another means. Have you considered using the "Manually Trusted Key Verification Strategy" to retrieve the remote host's key and then for you to verify the key against the contents of rsa_id.pub from the remote host? This should get round any delays with getting System Admins to perform actions on your behalf.
I needed this workaround from the mailing list:
> ssh -o HostKeyAlgorithms=ssh-rsa slave2.example.com
The implication below is that Jenkins is using weaker encryption.
https://groups.google.com/d/msgid/jenkinsci-users/7006ab93-7ca4-4063-baf6-7c844be60165%40googlegroups.com