-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
OS: centos
Jenkins: 2.223
OpenJDK 1.8.0_242
credentials-binding: 1.21
ssh-steps: 2.0.0
-
Powered by SuggestiMate
Hi.
I am currently working on a Jenkins pipeline and trying to move all credential handling in the domain of the jenkins instance.
To be able to execute an ssh command on another machine we use private key authentication. I tried to accomplish this by combining credentials-binding and ssh-steps but ran into a problem I couldn't find a solution to.
I basicly do this:
node { def remote = [:] remote.name = "integration_server" remote.host = integration_server remote.allowAnyHosts = true withCredentials([sshUserPrivateKey(credentialsId: 'myCredentialId', keyFileVariable: 'identity', passphraseVariable: '', usernameVariable: 'userName')]) { remote.user = userName remote.identityFile = identity sshCommand remote: remote, command: "do stuff" } ... withCredentials([sshUserPrivateKey(credentialsId: 'myCredentialId', keyFileVariable: 'identity', passphraseVariable: '', usernameVariable: 'userName')]) { remote.user = userName remote.identityFile = identity sshCommand remote: remote, command: "do some other stuff" } }
The first block runs fine. The second block fails with a FileNotFoundException looking for a temporary "secretFile".
My guess is, that credential-binding reuses the reference to the temporary file from the first block during execution of the second block while simultaniously deleting it after the first block ends.
I would like to be able to define multiple indipendent withCredential steps to omit wrapping the whole script in one large block.
[JENKINS-61341] FileNotFoundException if withCredentials([sshUserPrivateKey]) is called twice
theck / julian_alarcon
I tried to reproduce the issue but couldn't. Below is the code and I am on latest versions of both credentials binding (1.23) and ssh-steps plugin (2.0.0) and Jenkins master is on 2.230 (weekly)
I doubt if this is an issue with ssh steps plugin, looks like an issue with credentials-binding-plugin as mentioned it in the description, can you switch that sshCommand with readFile and verify
def fileName def remote = [:] remote.name = "integration_server" remote.user = "test" remote.host = "10.1.0.20" remote.allowAnyHosts = true node { stage('Stage A') { withCredentials([sshUserPrivateKey(credentialsId: 'serviceacctSSH', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { fileName = identityFileName def data = readFile(file: fileName) println data remote.identityFile = identityFileName sshCommand remote: remote, command: "ls -lrth" } } println fileName stage('Stage B') { withCredentials([sshUserPrivateKey(credentialsId: 'serviceacctSSH', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { fileName = identityFileName def data = readFile(file: fileName) println data remote.identityFile = identityFileName sshCommand remote: remote, command: "ls -lrth" } } println fileName }
I'm running credentials binding (1.23) and ssh-steps-plugin (2.0.0) on Jenkins LTS 2.222.3. This is my complete Declarative Pipeline, it stills fails inside in a different stage or even in the same Stage, I will try your scripted file.
def fileName def remote = [:] remote.name = "integration_server" remote.user = "ubuntu" remote.host = "10.0.0.15" remote.allowAnyHosts = truepipeline { agent any stages { stage('APP_01') { steps { withCredentials([sshUserPrivateKey(credentialsId: 'my-ssh-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { fileName = identityFileName def data = readFile(file: fileName) println data // <--- This print the secret ssh key with no issue sshCommand remote: remote, command: "echo first-app01-ok" } } withCredentials([sshUserPrivateKey(credentialsId: 'my-ssh-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { fileName = identityFileName def data = readFile(file: fileName) println data // <--- This print the secret ssh key again with no issues sshCommand remote: remote, command: "echo this-fails" } } } } stage('APP_01_AGAIN') { //this is never run, but if you comment the second withCredentials above it will run an fail steps { withCredentials([sshUserPrivateKey(credentialsId: 'my-ssh-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { fileName = identityFileName def data = readFile(file: fileName) println data remote.identityFile = identityFileName sshCommand remote: remote, command: "echo this-one-fails-too" } } } } } }
This is all my log execution
[2020-06-29T16:05:40.788Z] Started by user julian [2020-06-29T16:05:40.788Z] Running in Durability level: MAX_SURVIVABILITY [2020-06-29T16:05:40.821Z] [Pipeline] Start of Pipeline [2020-06-29T16:05:41.126Z] [Pipeline] node [2020-06-29T16:05:41.143Z] Running on Jenkins in /var/lib/jenkins/workspace/test-bug [2020-06-29T16:05:41.164Z] [Pipeline] { [2020-06-29T16:05:41.200Z] [Pipeline] stage [2020-06-29T16:05:41.204Z] [Pipeline] { (APP_01) [2020-06-29T16:05:41.248Z] [Pipeline] withCredentials [2020-06-29T16:05:41.253Z] Masking supported pattern matches of $identityFileName or $password or $username [2020-06-29T16:05:41.260Z] [Pipeline] { [2020-06-29T16:05:41.268Z] [Pipeline] script [2020-06-29T16:05:41.279Z] [Pipeline] { [2020-06-29T16:05:41.302Z] [Pipeline] readFile [2020-06-29T16:05:41.318Z] [Pipeline] echo [2020-06-29T16:05:41.322Z] -----BEGIN RSA PRIVATE KEY----- MY_SSH_KEY_PRINTED_WITH_NO_ISSUES [2020-06-29T16:05:41.323Z] -----END RSA PRIVATE KEY----- [2020-06-29T16:05:41.323Z] [2020-06-29T16:05:41.323Z] [2020-06-29T16:05:41.330Z] [Pipeline] sshCommand [2020-06-29T16:05:41.334Z] Executing command on integration_server[10.0.0.15]: echo first-app01-ok sudo: false [2020-06-29T16:05:41.557Z] first-app01-ok [2020-06-29T16:05:41.567Z] [Pipeline] } [2020-06-29T16:05:41.581Z] [Pipeline] // script [2020-06-29T16:05:41.588Z] [Pipeline] } [2020-06-29T16:05:41.604Z] [Pipeline] // withCredentials [2020-06-29T16:05:41.618Z] [Pipeline] withCredentials [2020-06-29T16:05:41.624Z] Masking supported pattern matches of $identityFileName or $password or $username [2020-06-29T16:05:41.631Z] [Pipeline] { [2020-06-29T16:05:41.638Z] [Pipeline] script [2020-06-29T16:05:41.646Z] [Pipeline] { [2020-06-29T16:05:41.663Z] [Pipeline] readFile [2020-06-29T16:05:41.682Z] [Pipeline] echo [2020-06-29T16:05:41.685Z] -----BEGIN RSA PRIVATE KEY----- MY_SSH_KEY_PRINTED_AGAIN_WITH_NO_ISSUES [2020-06-29T16:05:41.685Z] -----END RSA PRIVATE KEY----- [2020-06-29T16:05:41.685Z] [2020-06-29T16:05:41.685Z] [2020-06-29T16:05:41.693Z] [Pipeline] sshCommand [2020-06-29T16:05:41.697Z] Executing command on integration_server[10.0.0.15]: echo this-fails sudo: false [2020-06-29T16:05:41.715Z] [Pipeline] } [2020-06-29T16:05:41.732Z] [Pipeline] // script [2020-06-29T16:05:41.746Z] [Pipeline] } [2020-06-29T16:05:41.764Z] [Pipeline] // withCredentials [2020-06-29T16:05:41.778Z] [Pipeline] } [2020-06-29T16:05:41.794Z] [Pipeline] // stage [2020-06-29T16:05:41.816Z] [Pipeline] stage [2020-06-29T16:05:41.821Z] [Pipeline] { (APP_01_AGAIN) [2020-06-29T16:05:41.829Z] Stage "APP_01_AGAIN" skipped due to earlier failure(s) [2020-06-29T16:05:41.838Z] [Pipeline] } [2020-06-29T16:05:41.856Z] [Pipeline] // stage [2020-06-29T16:05:41.866Z] [Pipeline] } [2020-06-29T16:05:41.880Z] [Pipeline] // node [2020-06-29T16:05:41.903Z] [Pipeline] End of Pipeline [2020-06-29T16:05:41.931Z] java.io.FileNotFoundException: /var/lib/jenkins/workspace/test-bug@tmp/secretFiles/fce80e10-c180-44b7-a7ee-15919b0f3396/ssh-key-identityFileName (No such file or directory) [2020-06-29T16:05:41.931Z] at java.base/java.io.FileInputStream.open0(Native Method) [2020-06-29T16:05:41.931Z] at java.base/java.io.FileInputStream.open(FileInputStream.java:219) [2020-06-29T16:05:41.931Z] at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) [2020-06-29T16:05:41.931Z] at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112) [2020-06-29T16:05:41.931Z] at com.jcraft.jsch.Util.fromFile(Util.java:508) [2020-06-29T16:05:41.931Z] at com.jcraft.jsch.KeyPair.load(KeyPair.java:540) [2020-06-29T16:05:41.931Z] Caused: com.jcraft.jsch.JSchException [2020-06-29T16:05:41.931Z] at com.jcraft.jsch.KeyPair.load(KeyPair.java:543) [2020-06-29T16:05:41.931Z] at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40) [2020-06-29T16:05:41.931Z] at com.jcraft.jsch.JSch.addIdentity(JSch.java:406) [2020-06-29T16:05:41.931Z] at com.jcraft.jsch.JSch.addIdentity(JSch.java:387) [2020-06-29T16:05:41.931Z] at jdk.internal.reflect.GeneratedMethodAccessor2409.invoke(Unknown Source) [2020-06-29T16:05:41.931Z] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2020-06-29T16:05:41.931Z] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [2020-06-29T16:05:41.931Z] at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192) [2020-06-29T16:05:41.931Z] at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) [2020-06-29T16:05:41.931Z] at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper.configureUserAuthentication(UserAuthentication.groovy:36) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper$configureUserAuthentication$0.call(Unknown Source) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager.configureUserAuthentication(ConnectionManager.groovy) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.UserAuthentication$configureUserAuthentication$0.callCurrent(Unknown Source) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:104) [2020-06-29T16:05:41.933Z] at jdk.internal.reflect.GeneratedMethodAccessor2401.invoke(Unknown Source) [2020-06-29T16:05:41.933Z] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2020-06-29T16:05:41.933Z] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) [2020-06-29T16:05:41.933Z] at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384) [2020-06-29T16:05:41.933Z] at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy:85) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy) [2020-06-29T16:05:41.933Z] at jdk.internal.reflect.GeneratedMethodAccessor2400.invoke(Unknown Source) [2020-06-29T16:05:41.933Z] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2020-06-29T16:05:41.933Z] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) [2020-06-29T16:05:41.933Z] at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) [2020-06-29T16:05:41.933Z] at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.util.Utility.retry(Utility.groovy:52) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.util.Utility$retry.callStatic(Unknown Source) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:83) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy) [2020-06-29T16:05:41.933Z] at jdk.internal.reflect.GeneratedMethodAccessor2398.invoke(Unknown Source) [2020-06-29T16:05:41.933Z] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [2020-06-29T16:05:41.933Z] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) [2020-06-29T16:05:41.933Z] at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager.connect(ConnectionManager.groovy:59) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.connection.ConnectionManager$connect.call(Unknown Source) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.session.SessionTask.wetRun(SessionTask.groovy:61) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.session.SessionTask.call(SessionTask.groovy:48) [2020-06-29T16:05:41.933Z] at java_util_concurrent_Callable$call.call(Unknown Source) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.core.Service.run(Service.groovy:81) [2020-06-29T16:05:41.933Z] at org.hidetake.groovy.ssh.core.Service$run$0.call(Unknown Source) [2020-06-29T16:05:41.933Z] at org.jenkinsci.plugins.sshsteps.SSHService.executeCommand(SSHService.groovy:177) [2020-06-29T16:05:41.933Z] at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution$CommandCallable.execute(CommandStep.java:84) [2020-06-29T16:05:41.933Z] at org.jenkinsci.plugins.sshsteps.util.SSHMasterToSlaveCallable.call(SSHMasterToSlaveCallable.java:32) [2020-06-29T16:05:41.933Z] at hudson.remoting.LocalChannel.call(LocalChannel.java:45) [2020-06-29T16:05:41.933Z] at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution.run(CommandStep.java:72) [2020-06-29T16:05:41.933Z] at org.jenkinsci.plugins.sshsteps.util.SSHStepExecution.lambda$start$0(SSHStepExecution.java:84) [2020-06-29T16:05:41.933Z] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [2020-06-29T16:05:41.933Z] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [2020-06-29T16:05:41.933Z] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [2020-06-29T16:05:41.933Z] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [2020-06-29T16:05:41.933Z] at java.base/java.lang.Thread.run(Thread.java:834) [2020-06-29T16:05:41.933Z] Finished: FAILURE
Wow, same error with your code Naresh! I only changed the name of the ssh key, the IP and the username. The only difference is the Jenkins version (but both are really close) and maybe OS.
11:14:10 java.io.FileNotFoundException: /var/lib/jenkins/workspace/test-bug@tmp/secretFiles/891085a5-9260-4f75-9d91-ef4eb4d9c0ff/ssh-key-identityFileName (No such file or directory) 11:14:10 at java.base/java.io.FileInputStream.open0(Native Method) 11:14:10 at java.base/java.io.FileInputStream.open(FileInputStream.java:219) 11:14:10 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) 11:14:10 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112) 11:14:10 at com.jcraft.jsch.Util.fromFile(Util.java:508) 11:14:10 at com.jcraft.jsch.KeyPair.load(KeyPair.java:540) 11:14:10 Caused: com.jcraft.jsch.JSchException 11:14:10 at com.jcraft.jsch.KeyPair.load(KeyPair.java:543) 11:14:10 at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40) 11:14:10 at com.jcraft.jsch.JSch.addIdentity(JSch.java:406) 11:14:10 at com.jcraft.jsch.JSch.addIdentity(JSch.java:387) 11:14:10 at jdk.internal.reflect.GeneratedMethodAccessor2409.invoke(Unknown Source) 11:14:10 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 11:14:10 at java.base/java.lang.reflect.Method.invoke(Method.java:566) 11:14:10 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192) 11:14:10 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) 11:14:10 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) 11:14:10 at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper.configureUserAuthentication(UserAuthentication.groovy:36) 11:14:10 at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper$configureUserAuthentication$0.call(Unknown Source) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager.configureUserAuthentication(ConnectionManager.groovy) 11:14:10 at org.hidetake.groovy.ssh.connection.UserAuthentication$configureUserAuthentication$0.callCurrent(Unknown Source) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:104) 11:14:10 at jdk.internal.reflect.GeneratedMethodAccessor2401.invoke(Unknown Source) 11:14:10 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 11:14:10 at java.base/java.lang.reflect.Method.invoke(Method.java:566) 11:14:10 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) 11:14:10 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) 11:14:10 at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384) 11:14:10 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) 11:14:10 at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69) 11:14:10 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy:85) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy) 11:14:10 at jdk.internal.reflect.GeneratedMethodAccessor2400.invoke(Unknown Source) 11:14:10 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 11:14:10 at java.base/java.lang.reflect.Method.invoke(Method.java:566) 11:14:10 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) 11:14:10 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) 11:14:10 at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) 11:14:10 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) 11:14:10 at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) 11:14:10 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117) 11:14:10 at org.hidetake.groovy.ssh.util.Utility.retry(Utility.groovy:52) 11:14:10 at org.hidetake.groovy.ssh.util.Utility$retry.callStatic(Unknown Source) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:83) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy) 11:14:10 at jdk.internal.reflect.GeneratedMethodAccessor2398.invoke(Unknown Source) 11:14:10 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 11:14:10 at java.base/java.lang.reflect.Method.invoke(Method.java:566) 11:14:10 at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) 11:14:10 at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) 11:14:10 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager.connect(ConnectionManager.groovy:59) 11:14:10 at org.hidetake.groovy.ssh.connection.ConnectionManager$connect.call(Unknown Source) 11:14:10 at org.hidetake.groovy.ssh.session.SessionTask.wetRun(SessionTask.groovy:61) 11:14:10 at org.hidetake.groovy.ssh.session.SessionTask.call(SessionTask.groovy:48) 11:14:10 at java_util_concurrent_Callable$call.call(Unknown Source) 11:14:10 at org.hidetake.groovy.ssh.core.Service.run(Service.groovy:81) 11:14:10 at org.hidetake.groovy.ssh.core.Service$run$0.call(Unknown Source) 11:14:10 at org.jenkinsci.plugins.sshsteps.SSHService.executeCommand(SSHService.groovy:177) 11:14:10 at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution$CommandCallable.execute(CommandStep.java:84) 11:14:10 at org.jenkinsci.plugins.sshsteps.util.SSHMasterToSlaveCallable.call(SSHMasterToSlaveCallable.java:32) 11:14:10 at hudson.remoting.LocalChannel.call(LocalChannel.java:45) 11:14:10 at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution.run(CommandStep.java:72) 11:14:10 at org.jenkinsci.plugins.sshsteps.util.SSHStepExecution.lambda$start$0(SSHStepExecution.java:84) 11:14:10 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) 11:14:10 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 11:14:10 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 11:14:10 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 11:14:10 at java.base/java.lang.Thread.run(Thread.java:834) 11:14:10 Finished: FAILURE
I updated my Jenkins to 2.330, same error. I'm out of ideas.
This is my list of installed plugins:
ace-editor 1.1 true ansicolor 0.7.0 true antisamy-markup-formatter 2.0 true apache-httpcomponents-client-4-api 4.5.10-2.0 true authentication-tokens 1.4 true bouncycastle-api 2.18 true branch-api 2.5.6 true build-user-vars-plugin 1.5 true cloudbees-folder 6.14 true command-launcher 1.4 true credentials 2.3.10 true credentials-binding 1.23 true display-url-api 2.3.2 true docker-commons 1.16 true docker-workflow 1.23 true durable-task 1.34 true email-ext 2.69 true extended-read-permission 3.2 true git 4.3.0 true git-client 3.3.0 true git-server 1.9 true github 1.30.0 true github-api 1.114.3 true handlebars 1.1.1 true htmlpublisher 1.23 true jackson2-api 2.11.0 true jaxb 2.3.0.1 true jdk-tool 1.4 true jquery-detached 1.2.1 true jsch 0.1.55.2 true junit 1.29 true lockable-resources 2.8 true mailer 1.32 true matrix-auth 2.6.1 true matrix-project 1.16 true momentjs 1.1.1 true okhttp-api 3.14.9 true pipeline-build-step 2.12 true pipeline-config-history 1.6 true pipeline-graph-analysis 1.10 true pipeline-input-step 2.11 true pipeline-milestone-step 1.3.1 true pipeline-model-api 1.7.0 true pipeline-model-declarative-agent 1.1.1 true pipeline-model-definition 1.7.0 true pipeline-model-extensions 1.7.0 true pipeline-rest-api 2.13 true pipeline-stage-step 2.5 true pipeline-stage-tags-metadata 1.7.0 true pipeline-stage-view 2.13 true plain-credentials 1.7 true publish-over 0.22 true publish-over-ssh 1.20.1 true resource-disposer 0.14 true robot 2.1.1 true role-strategy 3.0 true saml 1.1.6 true scm-api 2.6.3 true script-security 1.73 true slack 2.40 true ssh-credentials 1.18.1 true ssh-steps 2.0.0 true structs 1.20 true timestamper 1.11.3 true token-macro 2.12 true trilead-api 1.0.8 true workflow-aggregator 2.6 true workflow-api 2.40 true workflow-basic-steps 2.20 true workflow-cps 2.80 true workflow-cps-global-lib 2.16 true workflow-durable-task-step 2.35 true workflow-job 2.39 true workflow-multibranch 2.21 true workflow-scm-step 2.11 true workflow-step-api 2.22 true workflow-support 3.5 true ws-cleanup 0.38 true
I am getting the same error. I am using Jenkins 2.235.3. Running Jenkins under windows 10 and agent under Centos 7.6.
Running my jobs all under the Centos agent, here is the log.
java.io.FileNotFoundException: C:/Jenkins/workspace/pipeline_test@tmp/secretFiles/a338eb6c-9e3e-4c8d-8adc-5d30f3be374a/ssh-key-keyFileName (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at java.io.FileInputStream.<init>(FileInputStream.java:93) at com.jcraft.jsch.Util.fromFile(Util.java:508) at com.jcraft.jsch.KeyPair.load(KeyPair.java:540) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Test_agent at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:998) at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution.run(CommandStep.java:72) at org.jenkinsci.plugins.sshsteps.util.SSHStepExecution.lambda$start$0(SSHStepExecution.java:84) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused: com.jcraft.jsch.JSchException at com.jcraft.jsch.KeyPair.load(KeyPair.java:543) at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40) at com.jcraft.jsch.JSch.addIdentity(JSch.java:406) at com.jcraft.jsch.JSch.addIdentity(JSch.java:387) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:192) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133) at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper.configureUserAuthentication(UserAuthentication.groovy:36) at org.hidetake.groovy.ssh.connection.UserAuthentication$Trait$Helper$configureUserAuthentication$2.call(Unknown Source) at org.hidetake.groovy.ssh.connection.ConnectionManager.configureUserAuthentication(ConnectionManager.groovy) at org.hidetake.groovy.ssh.connection.UserAuthentication$configureUserAuthentication$1.callCurrent(Unknown Source) at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190) at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy:85) at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117) at org.hidetake.groovy.ssh.util.Utility.retry(Utility.groovy:52) at org.hidetake.groovy.ssh.util.Utility$retry.callStatic(Unknown Source) at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:83) at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166) at org.hidetake.groovy.ssh.connection.ConnectionManager.connect(ConnectionManager.groovy:59) at org.hidetake.groovy.ssh.connection.ConnectionManager$connect$0.call(Unknown Source) at org.hidetake.groovy.ssh.session.SessionTask.wetRun(SessionTask.groovy:61) at org.hidetake.groovy.ssh.session.SessionTask.call(SessionTask.groovy:48) at java_util_concurrent_Callable$call$0.call(Unknown Source) at org.hidetake.groovy.ssh.core.Service.run(Service.groovy:81) at org.hidetake.groovy.ssh.core.Service$run$1.call(Unknown Source) at org.jenkinsci.plugins.sshsteps.SSHService.executeCommand(SSHService.groovy:177) at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution$CommandCallable.execute(CommandStep.java:84) at org.jenkinsci.plugins.sshsteps.util.SSHMasterToSlaveCallable.call(SSHMasterToSlaveCallable.java:32) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE
Let me know what can I do to help diagnose this issue. It is 100% reproducible.
Seems that I found the reason of the error. Every time that a new block withCredentials is invoked, a new temporal folder is created. But when a sshCommand or any other option from ssh-steps-plugin is used it locks the original temporal directory/file to be used, but as after every withCredentials run that file is deleted, it will not find it, ssh-steps-plugin should use the new temporal directory created and not be fixed with the first temporal path from the first withCredentials block.
I tested this code and also check the files, and checking the error that's what is happening.
Jenkinsfile code:
def fileName def remote = [:] remote.name = "integration_server" remote.user = "ubuntu" remote.host = "10.20.156.167" remote.allowAnyHosts = truepipeline { agent any stages { stage('APP_01') { steps { withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { fileName = identityFileName def data = readFile(file: fileName) println data // <--- This print the secret ssh key with no issues remote.identityFile = identityFileName sshCommand remote: remote, command: "echo first-app01-ok" } echo 'after first withCredentials block' sh 'sleep 30' } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { fileName = identityFileName def data = readFile(file: fileName) println data // <--- This print the secret ssh key again with no issues remote.identityFile = identityFileName //sshCommand remote: remote, command: "echo this-fails" } echo 'after second withCredentials block' sh 'sleep 30' } } } stage('APP_01_AGAIN') { //this is never run, but if you comment the second withCredentials above it will run an fail steps { withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { fileName = identityFileName def data = readFile(file: fileName) println data remote.identityFile = identityFileName sshCommand remote: remote, command: "echo this-one-fails-too" } } } } } }
Jenkins Error (Look, it's trying to find the 6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName but it doesn't exists anymore):
16:49:37 [Pipeline] sshCommand 16:49:37 Executing command on integration_server[10.20.156.167]: echo this-one-fails-too sudo: false 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // script 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // withCredentials 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // stage 16:49:37 [Pipeline] } 16:49:37 [Pipeline] // node 16:49:37 [Pipeline] End of Pipeline 16:49:37 java.io.FileNotFoundException: /var/lib/jenkins/workspace/test-bug-JENKINS-61341@tmp/secretFiles/6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName (No such file or directory) 16:49:37 at java.base/java.io.FileInputStream.open0(Native Method) 16:49:37 at java.base/java.io.FileInputStream.open(FileInputStream.java:219) 16:49:37 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) 16:49:37 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112) 16:49:37 at com.jcraft.jsch.Util.fromFile(Util.java:508) 16:49:37 at com.jcraft.jsch.KeyPair.load(KeyPair.java:540) 16:49:37 Caused: com.jcraft.jsch.JSchException 16:49:37 at com.jcraft.jsch.KeyPair.load(KeyPair.java:543) 16:49:37 at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40) 16:49:37 at com.jcraft.jsch.JSch.addIdentity(JSch.java:406) 16:49:37 at com.jcraft.jsch.JSch.addIdentity(JSch.java:387) 16:49:37 at jdk.internal.reflect.GeneratedMethodAccessor4870.invoke(Unknown Source) 16:49:37 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Checking the real files in the server, the files with the key exists but every time than a withCredentials block is used it deletes the files and recreates them, with another tmp name.
jenkins@ip-10-20-150-196:~/workspace/test-bug-JENKINS-61341@tmp$ cat secretFiles/6e2a9038-339c-4988-a5a5-4468bacea4c8/ssh-key-identityFileName -----BEGIN RSA PRIVATE KEY----- MY_SSH_KEY . . . -----END RSA PRIVATE KEY----- jenkins@ip-10-20-150-196:~/workspace/test-bug-JENKINS-61341@tmp$ cat secretFiles/4b169093-f5a3-4316-b7e2-e940149b361f/ssh-key-identityFileName -----BEGIN RSA PRIVATE KEY----- MY_SSH_KEY . . . -----END RSA PRIVATE KEY-----
Can you take a look nrayapati?
That looks like a good find julian_alarcon, please let me know if I can help in anything. I am using a scripted pipeline.
Hi, as I found the reason (not changing the value of the map value remote.identityFile), I was able to repeat the execution using two workarounds, you can find my code here:
server_01 = [:] server_01.name = "integration_server" server_01.host = "10.20.11.1" def remotename = "integration_server" def remotehost = "10.20.11.1" pipeline { agent any stages { stage('APP_01') { steps { withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { def remote = [ name: server_01.name, host: server_01.host, user: username, identityFile: identityFileName, allowAnyHosts: true] sshCommand remote: remote, command: "echo First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block" } } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { sshCommand remote: [ name: remotename, host: remotehost, allowAnyHosts: true, user: username, identityFile: identityFileName ], command: "echo works too, but seems ugly, as it is too large" } } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { def remote = [ name: server_01.name, host: server_01.host, user: username, identityFile: identityFileName, allowAnyHosts: true] sshCommand remote: remote, command: "echo First workaround. It keeps working" } } withCredentials([sshUserPrivateKey(credentialsId: 'shared-dev-key', keyFileVariable: 'identityFileName', passphraseVariable: 'password', usernameVariable: 'username')]) { script { sshCommand remote: [ name: remotename, host: remotehost, allowAnyHosts: true, user: username, identityFile: identityFileName ], command: "echo Second workaround. It keeps working too" } } } } } }
Job Log:
Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /var/lib/jenkins/workspace/test-bug-JENKINS-61341 [Pipeline] { [Pipeline] stage [Pipeline] { (APP_01) [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.11.1]: echo First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block sudo: false First workaround setting a new map for the withCredentials step It is possible to repeat the same map name remote as it is defined only by script block [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.156.167]: echo works too, but seems ugly, as it is too large sudo: false works too, but seems ugly, as it is too large [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.11.1]: echo First workaround. It keeps working sudo: false First workaround. It keeps working [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] withCredentials Masking supported pattern matches of $identityFileName or $password or $username [Pipeline] { [Pipeline] script [Pipeline] { [Pipeline] sshCommand Executing command on integration_server[10.20.156.167]: echo Second workaround. It keeps working too sudo: false Second workaround. It keeps working too [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS
Hi!
I think that this is a similar issue of this one, but not the same: https://issues.jenkins-ci.org/browse/JENKINS-54357
BTW I also have this issue using Declarative Pipeline Syntax
This is my code:
The problem seems to be present when you are using the same credentialsId with the same remote server (server_01 in my case) for ssh-steps-plugin, as I'm using the same credentialsId but with other remote server definition with no issues.
This is the error:
15:51:29 java.io.FileNotFoundException: /var/lib/jenkins/workspace/my-job@tmp/secretFiles/32f06132-563a-492d-aabb-dfgdgf34df/ssh-key-SSH_KEY (No such file or directory) 15:51:29 at java.base/java.io.FileInputStream.open0(Native Method) 15:51:29 at java.base/java.io.FileInputStream.open(FileInputStream.java:219) 15:51:29 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) 15:51:29 at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)