-
Improvement
-
Resolution: Unresolved
-
Minor
-
None
-
Powered by SuggestiMate
It would be really nice if we could use the Vault tokens not just during the build process but also to get authentication tokens for the SCM stage. That way we would only have to provide jenkins with the ability to get tokens from Vault and can store all our other tokens in Vault which provides the ability to have tokens expire etc. etc.
- is related to
-
JENKINS-60440 Invalid git username/password on Jenkins agent when using Vault Username-Password Credential with '@' in username
-
- Open
-
[JENKINS-39374] Add ability to get SCM authentication tokens from Vault
rvodden if you do let me know if you need a tester. I would love to have this! Removing the credentials from Jenkins will go a long way towards making jenkins easier to deploy from a script / configuration management system.
Is there any partially completed work on this? Is there any active work being done right now? We can help if needed.
We'd like to use this to auth the github organization plugin, the kubernetes plugin, etc. and would allow us to have a solid security posture on jenkins. If this were a plugin that exposed a credential provider (not a sync to the jenkins credential store) then the only credential we would need in the jenkins credential store is the one used by the vault plugin. In the case of an out of band verified auth method such as jwt based ones like gcp auth or k8s auth the value of the credential is not sensitive. This way no actual sensitive information is stored in Jenkins but rather just exposed to jobs ephemerally via this vault plugin.
sanbornick I haven't seen anything. Above rvodden said he was going to put together a PR but I haven't seen one yet. I would love to have the ability to use Vault to handle all the credentials. That would reduce the attack surface for all the connected services
I did have a look at this, and then real life got in the way. Let me have a look at how far I got and at the very least push a branch up or something.
We wouldn't be able to put effort towards this for at least a month but if you pushed up your branch with whatever you have that would be helpful for estimation.
H'ok - I've found the work I did (two laptops ago) and have had a bit of a look over the weekend. The progress I've made is here (not in a PR state yet) - i'll be putting some work into this over the next few days so hopefully, it will be ready for a proper review quite soon - structural/approach opinions are welcome immediately
https://github.com/rvodden/hashicorp-vault-plugin/tree/feature/JENKINS-39374
A few contextual comments:
1) There are three approaches to implementing a CredentialsProvider which is backed by an external party system. Read/Write, where creds can be entered into Jenkins and persisted by the external system; Read-Only implicit (used by AWS and Kubernetes plugins) where, in effect, every secret within the external system is presented as a secret in Jenkins; and Read-Only explicit where a consumer must explicitly pull in secrets from the external system by adding them in the CredentialsStore or through, for example, a build wrapper. I have initially gone with the 3rd way as Vault doesn't provide a straightforward way of listing all the credentials it's holding - and in the case of dynamic back ends listing the secrets makes no sense.
2) As you'll see, I've effectively implemented an entirely new plugin in the .provider package. This was originally to keep the proof of concept code apart from the production code, with a view to merging it in later, however, the implementation is necessarily quite different, so we may decide to just leave it there.
3) Using credential type to indicate the vault authentication method which should be used provided me with a couple of challenges:
- Credentials have to be provided directly via the UI or CasC, they can't be consumed from an alternative credentials provider.
- The UI became a bit odd as it was difficult to distinguish the credentials used to configure vault authentication versus the credentials provided by the vault plugin.
I therefore suggest and have implemented, that an authenticator interface is implemented for each authentication approach, with a matching authentication configuration implementation which provides the necessary data. It is therefore up to the individual implementations of the authenticator and authentication configuration where the data is sourced from. The user experience in configuring the plugin is very similar - I have implemented Token authentication by way of example. Simply select the authentication method and then provide the necessary data - only now it is possible to select any string credential as the token - meaning that this could be provided by the AWS Secret Manager plugin, or even from another Vault instance (see below). I'll add another couple to further illustrate the approach. (I've prefixed the credential types which are used to configure vault authentication with (OLD) - this is for my own sanity whilst implementing, and not a serious suggestion )
4) Right now I've only implemented GLOBAL scope. This is fine for the moment as generic build wrappers can pull in credentials which have been configured globally (so works for PoC) but if we want to provide path and key information to a build wrapper to instantiate a secret, which I can see being a use case, then we'll at least need job level scope. I don't think this is hard so will try and include this in this week's work.
5) I'm trialing an approach which permits more than one vault instance to be consumed. By instance, I mean a unique pair of URL and authentication; so in theory two instances could be the same instance but with alternative credentials. I can see this being quite helpful but right now I'm having trouble working out the best place to persist the map of vault instances. At present it's in the VaultCredentialsProvider which implements savable, (but I've not yet tested if it's actually saved), this presents a small challenge in getting the vault instance to the credential - as the credential has no knowledge of the provider which provided it. I'll do some reading around the various persistence options within Jenkins and come up with something.
6) I'm not clear if a CredentialStore is a good idea. The [credentials plugin implementation guide|https://github.com/jenkinsci/credentials-plugin/blob/master/docs/implementation.adoc] says that "The main work in an implementation will be the mapping to CredentialStore instances. Any "explicitly exposed" style implementation will have CredentialsStore instance for each context that persists the IDs of the credentials to be exposed and the credentials domains with which those credentials are to be associated." This implies that by design there should be a credential store for each CredentialProvider, so that's the route I'm going (which is also the route followed by the AWS and Kubernetes plugins); however at this stage I'm not clear on why the creds can't be persisted in the existing CredentialStores.
I'm quite pleased with how this has gone so far, but as you can see there's still quite a lot to do - thoughts comments and suggestions are very much welcome!
casz Have you been following this conversation? It would be really good to get your input
I don't want yet another plugin for vault
I would very much like to have Vault Plugin to become a credential provider since it should simplify some of the work that we need to get done merging in https://github.com/jenkinsci/hashicorp-vault-pipeline-plugin
From my understanding it would be best to implement a CredentialStore but prevent adding/updating/deleting
Once you created a credential store we can control what kind of Credential it can store. I would say we should avoid implementing caching. That's what Vault Agent was created for
Okay, credential store it is. What about the vault authentication itself? Using specific credentials for this is quite restrictive, makes for a confusing UI (see above) and isn't a pattern adopted by any other credential provider. The approach I've coded means that any credential provider can be used to provide the authentication information for vault, but it is a breaking change.
I'd be really interested in helping with this. Exposing the vault secrets via a CredentialsProvider will make for a much more flexible plugin. I'll knock together a PR unless someone shouts.