-
Bug
-
Resolution: Fixed
-
Critical
-
None
Some properties of a credential may need to be resolved at point of use rather than being stored in the credential itself.
Observed the following as an example stacktraces:
With JGit as the client:
java.lang.NullPointerException at jenkins.security.ConfidentialStore.get(ConfidentialStore.java:65) at jenkins.security.ConfidentialKey.load(ConfidentialKey.java:46) at jenkins.security.CryptoConfidentialKey.getKey(CryptoConfidentialKey.java:32) at jenkins.security.CryptoConfidentialKey.decrypt(CryptoConfidentialKey.java:67) at hudson.util.Secret.decrypt(Secret.java:151) at hudson.util.Secret.fromString(Secret.java:200) at REDACTED.getPassword(REDACTED.java:136) at org.jenkinsci.plugins.gitclient.trilead.SmartCredentialsProvider.get(SmartCredentialsProvider.java:132) at org.eclipse.jgit.transport.HttpAuthMethod.authorize(HttpAuthMethod.java:219) at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:502) at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:309) at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136) at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122) at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1138) at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:130) at org.jenkinsci.plugins.gitclient.JGitAPIImpl$5.execute(JGitAPIImpl.java:1448) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:332) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at ......remote call to mac-os(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:781) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131) at com.sun.proxy.$Proxy133.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1046) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at hudson.scm.SCM.checkout(SCM.java:495) at hudson.model.AbstractProject.checkout(AbstractProject.java:1269) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1741) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410)
with CLI Git as the client
FATAL: null java.lang.NullPointerException at jenkins.security.ConfidentialStore.get(ConfidentialStore.java:65) at jenkins.security.ConfidentialKey.load(ConfidentialKey.java:46) at jenkins.security.CryptoConfidentialKey.getKey(CryptoConfidentialKey.java:32) at jenkins.security.CryptoConfidentialKey.decrypt(CryptoConfidentialKey.java:67) at hudson.util.Secret.decrypt(Secret.java:151) at hudson.util.Secret.fromString(Secret.java:200) at REDACTED.getPassword(REDACTED.java:136) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getGitCredentialsURL(CliGitAPIImpl.java:2635) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1383) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:506) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:332) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at ......remote call to mac-os(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:781) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131) at com.sun.proxy.$Proxy133.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1046) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1086) at hudson.scm.SCM.checkout(SCM.java:495) at hudson.model.AbstractProject.checkout(AbstractProject.java:1269) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1741) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) Finished: FAILURE
In this case the REDACTED class was attempting to make a call to retrieve the password from an external password store... but that call cannot take place when working on a remote system... so in that case it returns null as the password string that then has to be encrypted into a Secret...
Now there seems to be another issues here also... Namely a remote agent can never decrypt a Secret so that is a set-up for failure...
So I think what should be happening is that the SmartCredentialsProvider should be Channel.exported() rather than transferred over the wire... that way it will access the credentials on the master and perform the password decryption on the one node that can decrypt a secret... then the plain text can be sent to the remote node (of course for security you want to ensure you are using either SSH agents or JNLP protocol 3/4+)... that removes the need to CredentialsProvider.snapshot() and should fix issues.
Similarly CliGitAPIImpl.getGitCredentialsURL seems to have the same broken assumption.
I suspect also that I have hit upon the root cause as to Password protected SSH Keys not working with the Git on remote agents... namely that the getPassword().getPlainText() is being called on the remote agent and not on the master... and hence the password cannot be decrypted and hence the ssh key cannot be used.
[JENKINS-37899] Git client does not call CredentialsProvider.snapshot() when adding credentials to a SmartCredentialsProvider that will be used on a remote instance
Pull Request submitted: https://github.com/jenkinsci/git-client-plugin/pull/235
I'm still unable to make the git plugin work with a passphrase protected ssh private key on a local or a remote agent.
Were you able to confirm that the code was not working before your change, and that it is now working after your change?
Can you provide more details of the configuration steps you're using?
> I'm still unable to make the git plugin work with a passphrase protected ssh private key on a local or a remote agent.
Here are my tests, is it what you mean?
| SSH Key | Build on | Result |
|---|---|---|
| Private Key=Enter directly, passphrase=no | master |  |
| Private Key=Enter directly, passphrase=yes | master | |
| Private Key=Enter directly, passphrase=yes | remote build agent via ssh | |
| Private Key=From a file on Jenkins master, passphrase=yes | remote build agent via ssh | |
| Private Key=From a file on Jenkins master, passphrase=no | remote build agent via ssh | |
Environment
Git client plugin (git-client): 2.3.0-SNAPSHOT -> latest from github Ant Plugin (ant): 1.2 OWASP Markup Formatter Plugin (antisamy-markup-formatter): 1.1 Credentials Plugin (credentials): 2.1.12 CVS Plug-in (cvs): 2.11 Display URL API (display-url-api): 1.1.1 External Monitor Job Type Plugin (external-monitor-job): 1.4 GIT server Plugin (git-server): 1.7 Git plugin (git): 3.0.5 Javadoc Plugin (javadoc): 1.1 JUnit Plugin (junit): 1.20 LDAP Plugin (ldap): 1.11 Mailer Plugin (mailer): 1.19 Matrix Authorization Strategy Plugin (matrix-auth): 1.1 Matrix Project Plugin (matrix-project): 1.8 Maven Integration plugin (maven-plugin): 2.7.1 PAM Authentication plugin (pam-auth): 1.1 SCM API Plugin (scm-api): 2.0.7 Script Security Plugin (script-security): 1.13 SSH Credentials Plugin (ssh-credentials): 1.12 SSH Slaves plugin (ssh-slaves): 1.9 Structs Plugin (structs): 1.5 Subversion Plug-in (subversion): 1.54 Translation Assistance plugin (translation): 1.10 Windows Slaves Plugin (windows-slaves): 1.0 Pipeline: SCM Step (workflow-scm-step): 1.14.2 Pipeline: Step API (workflow-step-api): 1.14.2
Code changed in jenkins
User: Cyrille Le Clerc
Path:
src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java
http://jenkins-ci.org/commit/git-client-plugin/0381412bdd3ea7639cc6cdbb708fd03dc6a38339
Log:
JENKINS-37899 RemoteGitImpl: Take a snapshot of the credentials before passing them to the git client proxy
We need to snapshot of the credentials before passing them to the git client proxy so that the credentials are snapshotted before being serialised and transferred to the build agent through Jenkins remoting.
Code changed in jenkins
User: Mark Waite
Path:
src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java
http://jenkins-ci.org/commit/git-client-plugin/e75b299c645b2155f693897754bb6a16501a3117
Log:
Merge pull request #235 from cyrille-leclerc/JENKINS-37899
JENKINS-37899 RemoteGitImpl: Take a snapshot of the credentials before passing them to the git client proxy
Compare: https://github.com/jenkinsci/git-client-plugin/compare/196ed5dbb394...e75b299c645b
Change will be included in git client plugin 2.3.0, slated to be released within the next week
Thanks for the insight. Passphrase protected private keys in the git plugin have been a source of puzzlement to me for a very long time. There are several bug reports that they don't work in various ways. Oddly, I have a docker instance which I run in two very different environments where passphrase protected private keys work on slaves, but don't work on the master (of that docker instance). I've spent time creating various intentionally interesting ssh passphrases (trivial characters, interesting special characters, etc.), yet still haven't understood the failure cases well enough.
I can't explain why my passphrase protected slave cases work, since I think they are the precise case you're describing should fail. I'll investigate further while evaluating git client PR207