-
Bug
-
Resolution: Cannot Reproduce
-
Minor
-
None
Using Consul as a secret backend, the plugin fails to fetch data as it tries to make a single API call fetching a list of secrets at each path.
Example:
/secret/test/stuff contains thing1 and thing2. If API requests were made to /secret/test/stuff/thing1 and /secret/test/stuff/thing2 they would succeed, but the plugin tries to make a single request to /secret/test/stuff and iterate over the data returned.
With consul as a backend, this single request returns a 403.
[JENKINS-38941] Cannot retrieve secrets if vault backend doesn't support fetching lists of secrets
Upon further investigation, we might have a very restrictive policy in place that is not allowing lists of keys to be returned. Just mentioning as it might not be specific to Consul backends. I don't have access to the configuration of Vault being used here, so i'm unable to dig into the configutation too much.
I also found a workaround - setting the secret path to a single key and only defining one value in the pair does succeed, though it is more verbose.
Gareth Harcombe-Minson
added a comment - - edited Upon further investigation, we might have a very restrictive policy in place that is not allowing lists of keys to be returned. Just mentioning as it might not be specific to Consul backends. I don't have access to the configuration of Vault being used here, so i'm unable to dig into the configutation too much.
I also found a workaround - setting the secret path to a single key and only defining one value in the pair does succeed, though it is more verbose. 
Peter Tierno
added a comment - garadox I was not able to reproduce this. I tested a two node vault cluster in HA mode with a 3 node consul cluster as the storage backend.
Peter Tierno
added a comment - garadox I was not able to reproduce this. I tested a two node vault cluster in HA mode with a 3 node consul cluster as the storage backend. Gareth Harcombe-Minson
added a comment - Something tells me it's a specific configuration we have on preventing non admin's from listing secrets. I'm asking our team that manages it for the specific policy preventing it.
If it is a configuration restriction, would you be interested in changing the underlying code to attempt individual key fetching in the event of a 403 on fetching the list?
Gareth Harcombe-Minson
added a comment - Something tells me it's a specific configuration we have on preventing non admin's from listing secrets. I'm asking our team that manages it for the specific policy preventing it.
If it is a configuration restriction, would you be interested in changing the underlying code to attempt individual key fetching in the event of a 403 on fetching the list? I wouldn't mind implementing something like that. Error checking has proved challenging in general when using the vault-java-driver. In a future iteration I may implement custom vault client code for the purposes of jenkins integration. Once I have some time I will try and get something implemented. There are a couple of other issues that have a higher priority that I need to find time for. I am always open to PR's though 
Peter Tierno
added a comment - - edited I wouldn't mind implementing something like that. Error checking has proved challenging in general when using the vault-java-driver. In a future iteration I may implement custom vault client code for the purposes of jenkins integration. Once I have some time I will try and get something implemented. There are a couple of other issues that have a higher priority that I need to find time for. I am always open to PR's though Peter Tierno
added a comment - After a quick thought I'm starting to think that working around the 403 may not be the best approach overall. Maybe an optional config param for each secret to enable that behaviour would be better. something like a 'fail on error' option.
Peter Tierno
added a comment - After a quick thought I'm starting to think that working around the 403 may not be the best approach overall. Maybe an optional config param for each secret to enable that behaviour would be better. something like a 'fail on error' option. Gareth Harcombe-Minson
added a comment - After looking at the source of the two projects, the best option might be to keep my workaround and specify the full path to the key in the build job. When you have such a restriction, i'm not sure either project could "do the right thing" and fetch the data appropriately.
Gareth Harcombe-Minson
added a comment - After looking at the source of the two projects, the best option might be to keep my workaround and specify the full path to the key in the build job. When you have such a restriction, i'm not sure either project could "do the right thing" and fetch the data appropriately. Peter Tierno
added a comment - Closing this as I cannot reproduce. Appears to be an issue with the reporters consul acl's.
Peter Tierno
added a comment - Closing this as I cannot reproduce. Appears to be an issue with the reporters consul acl's. 
garadox Thanks for reporting this. I will look into this.