Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44772

User Scoped credentials are not used by the "withCredentials" pipeline step

      Environment

      • Jenkins 2.46.2
      • credentials:2.1.13 'Credentials Plugin'
      • credentials-binding:1.11 'Credentials Binding Plugin'
      • workflow-aggregator:2.5 'Pipeline'
      • workflow-api:2.12 'Pipeline: API'
      • workflow-basic-steps:2.4 'Pipeline: Basic Steps'
      • workflow-cps:2.29 'Pipeline: Groovy'
      • workflow-cps-checkpoint:2.4 'CloudBees Pipeline: Groovy Checkpoint Plugin'
      • workflow-cps-global-lib:2.7 'Pipeline: Shared Groovy Libraries'
      • workflow-durable-task-step:2.10 'Pipeline: Nodes and Processes'
      • workflow-job:2.10 'Pipeline: Job'
      • workflow-multibranch:2.14 'Pipeline: Multibranch'
      • workflow-scm-step:2.4 'Pipeline: SCM Step'
      • workflow-step-api:2.9 'Pipeline: Step API'
      • workflow-support:2.14 'Pipeline: Supporting APIs'

      Scenario to Use User Scoped Credentials in "withCredentials()"

      Many organizations use GPG Signing Key and special permissions on Nexus / Artifactory to create releases. For traceability and security, these privileged credentials may be managed as are "per individual/personal credentials", they may not be shared with team members.

      For this kind of credentials, we want to use Jenkins User Scoped Credentials in pipeline (withCredentials, git, config-file-provider, ssh-agent...)

      Description

      When using the authorize project plugin,

      • User Scoped Credentials are not found by the "withCredentials" pipeline step.
      • Global Credentials overwritten by user scoped credentials are not overwritten when used with the "withCredentials" pipeline step.

      Reproduce

      • Install the Project Authorize Plugin and configure it "Run as user who triggered the build"
      • Create a global credential "global-credentials-intended-to-be-overwritten-at-the-user-scope"
      • Create user scoped credentials "global-credentials-intended-to-be-overwritten-at-the-user-scope"
      • create a pipeline with "withCredentials" binding 'global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope' and writing it in a text file
      • run the build, open the text file in the workspace and verify that the global credentials are NOT overwritten
      • Create user scoped credentials "my-username-password"
      • Create a pipeline with "withCredentials" and the "my-username-password" credentials
      • job will fail with "CredentialNotFoundException"
      node {
          // verify that the build is properly impersonated by the https://wiki.jenkins-ci.org/display/JENKINS/Authorize+Project+plugin
          echo "Build is running as user " + org.acegisecurity.context.SecurityContextHolder.getContext().getAuthentication().toString()
          
          stage ("Global Credentials Overwritten at the user scope") {
              // credentials declared globally and overwritten by a user scoped credentials
              withCredentials([
                  usernamePassword(
                      credentialsId: 'global-credentials-intended-to-be-overwritten-at-the-user-scope', 
                      passwordVariable: 'PASSWORD_VAR', 
                      usernameVariable: 'USERNAME_VAR')]) {
                 sh "echo $PASSWORD_VAR > spy-overwritten-creds.txt"
              }
          }
          stage ("User Scoped Credentials") {
              withCredentials([
                  usernamePassword(
                      credentialsId: 'my-username-password', 
                      passwordVariable: 'PASSWORD_VAR', 
                      usernameVariable: 'USERNAME_VAR')]) {
                 sh "echo $PASSWORD_VAR > spy-user-scoped-credentials.txt"
              }
          }
      }
      
      Started by user admin
      [Pipeline] node
      Running on agent-1 in /home/ubuntu/jenkins-aws-home/workspace/tests/user-scoped-credentials-pipeline-step-with-credentials
      [Pipeline] {
      [Pipeline] echo
      Build is running as user org.acegisecurity.providers.UsernamePasswordAuthenticationToken@965748a4: Username: admin; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: authenticated
      [Pipeline] stage
      [Pipeline] { (Global Credentials Overwritten at the user scope)
      [Pipeline] withCredentials
      [Pipeline] {
      [Pipeline] sh
      [user-scoped-credentials-pipeline-step-with-credentials] Running shell script
      + echo ****
      [Pipeline] }
      [Pipeline] // withCredentials
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (User Scoped Credentials)
      [Pipeline] withCredentials
      [Pipeline] // withCredentials
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      org.jenkinsci.plugins.credentialsbinding.impl.CredentialNotFoundException: my-username-password
        at org.jenkinsci.plugins.credentialsbinding.MultiBinding.getCredentials(MultiBinding.java:153)
        at org.jenkinsci.plugins.credentialsbinding.impl.UsernamePasswordMultiBinding.bind(UsernamePasswordMultiBinding.java:76)
        at org.jenkinsci.plugins.credentialsbinding.impl.BindingStep$Execution.start(BindingStep.java:114)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:184)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:126)
        at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
        at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
        at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151)
        at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:21)
        at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:115)
        at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
        at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16)
        at WorkflowScript.run(WorkflowScript:16)
        at ___cps.transform___(Native Method)
        at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
        at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
        at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
        at sun.reflect.GeneratedMethodAccessor591.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
        at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
        at com.cloudbees.groovy.cps.Next.step(Next.java:74)
        at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
        at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
        at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:165)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:328)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228)
        at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
        at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
      Finished: FAILURE
      

          [JENKINS-44772] User Scoped credentials are not used by the "withCredentials" pipeline step

          Thanks pskopek for this PR, what is the actual state of this issue/PR?

          We have a similar problem, because we run deployments based to the user scoped credentials and this don't work since a while.
          So we would like to see this PR merged. 

          Matthias Baldi added a comment - Thanks pskopek for this PR, what is the actual state of this issue/PR? We have a similar problem, because we run deployments based to the user scoped credentials and this don't work since a while. So we would like to see this PR merged. 

          John Zerbe added a comment -

          Are there any updates for this issue?

          John Zerbe added a comment - Are there any updates for this issue?

          Any progress or any workaround? We want to use user scope credentials to manage releases and it is not possible to use on pipelines.

          Fernando Rosado Altamirano added a comment - Any progress or any workaround? We want to use user scope credentials to manage releases and it is not possible to use on pipelines.

          I will clarify, i could use user scope credential using the workarround commented on following issue ticket :
          https://issues.jenkins-ci.org/browse/JENKINS-38963?focusedCommentId=318189&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-318189

          The use of the single quotes around '${credential_id}' is mandatory, it is not the same issue commented (overwrite a credentialID) but perhaps exists some relation.

          Fernando Rosado Altamirano added a comment - I will clarify, i could use user scope credential using the workarround commented on following issue ticket : https://issues.jenkins-ci.org/browse/JENKINS-38963?focusedCommentId=318189&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-318189 The use of the single quotes around '${credential_id}' is mandatory, it is not the same issue commented (overwrite a credentialID) but perhaps exists some relation.

          Matt Sicker added a comment -

          Sounds like this feature may be implemented by JENKINS-58170? It works for withCredentials at least, though I've found other plugins that don't work properly with this new feature.

          Matt Sicker added a comment - Sounds like this feature may be implemented by JENKINS-58170 ? It works for withCredentials at least, though I've found other plugins that don't work properly with this new feature.

          Vít Zikmund added a comment -

          Hello jvz, I'm quite sure it is not the case. Your JENIKINS-58170 is nice, but it doesn't allow you to access the userscope credentials without them being supplied in a parameter.
          I suppose cleclerc was trying to avoid using the build parameter entirely.

          I personally don't see the security gain in this, especially if it's me, who's running the job, why wouldn't I want the job to access my user credentials? I know there's a possibility that if this got enabled, the job could guess and snoop any user's credential, but there could also be a way for the job to declare which credentials it's about to be accessing which the user would (until further change) accept only once and use it without selecting a credentials parameter each time s/he runs the build.

          I actually found a way how to circumvent the limitation by creating an upstream job, that uses a user's jenkins API token to trigger the job with credentials via the Jenkins API (so the downstream build thinks it was directly run by the user, which is one of the requirements for the user creds to be read), but it required me/my peers to reinvent the whole build() logic with curl, and that just sucked. I'd still give an eternal praise to someone willing to reconsider these limitations to make them more user friendly.

          Vít Zikmund added a comment - Hello jvz , I'm quite sure it is not the case. Your JENIKINS-58170 is nice, but it doesn't allow you to access the userscope credentials without them being supplied in a parameter. I suppose cleclerc was trying to avoid using the build parameter entirely. I personally don't see the security gain in this, especially if it's me, who's running the job, why wouldn't I want the job to access my user credentials? I know there's a possibility that if this got enabled, the job could guess and snoop any user's credential, but there could also be a way for the job to declare which credentials it's about to be accessing which the user would (until further change) accept only once and use it without selecting a credentials parameter each time s/he runs the build. I actually found a way how to circumvent the limitation by creating an upstream job, that uses a user's jenkins API token to trigger the job with credentials via the Jenkins API (so the downstream build thinks it was directly run by the user, which is one of the requirements for the user creds to be read), but it required me/my peers to reinvent the whole build() logic with curl, and that just sucked. I'd still give an eternal praise to someone willing to reconsider these limitations to make them more user friendly.

          Matt Sicker added a comment -

          The parameter API is the high level API. You can attach the actions directly via the low level API to simulate credential authorization by a user.

          Matt Sicker added a comment - The parameter API is the high level API. You can attach the actions directly via the low level API to simulate credential authorization by a user.

          Matt Sicker added a comment -

          vit_zikmund: are you using authorize project? If the build is running as the SYSTEM user, then it won't have access to other user's credentials as it's, you know, a different user.

          Matt Sicker added a comment - vit_zikmund : are you using authorize project? If the build is running as the SYSTEM user, then it won't have access to other user's credentials as it's, you know, a different user.

          Vít Zikmund added a comment - - edited

          Hi jvz, here's what we deal with. First of all, we do use the Authorize Project Plugin with all jobs set to: Run as the user who triggered the build.

          There's one "master" job with a credential parameter. Via that one, we can access the selected user-scope credential and all is nice and smooth.

          The problem is, that this job triggers another builds via the build() step and those builds need the "master" build credential as well. And that's where we hit the wall, or mud at least, as there's an ugly way through, but the credentials parameter is not a big help in either of the workarounds we've figured out:

          1. Use the credentials parameter in the downstream jobs and trigger them via Jenkins API and the user's auth token to pretend it was directly him/her who did it.
            This allows the authorize/credentials plugins to work properly and reach for the user's creds.
            The downside is that the triggering from the "master" job can't be done with the build() step, as that puts the master build as the main trigger cause, which IMHO kills the authorize plugin's Run as the user who triggered the build (even though the original user-triggered cause is also transitively visible)
            In one case we really reinvented the build() wheel to do that, and we made the users to save their auth tokens in their user creds and trigger the "master" build with it.
            For illustration the core looks like this:
            curl -sS -f -i -X POST "https://$JENKINS_HOSTNAME/job/$JOBNAME/buildWithParameters" --user "$USERNAME:$TOKEN" -data-urlencode "user_cred=$DESIRED_USER_CRED_ID"
            Sad things:
          • rewritten build() in "user space"
          • user has to manage one more credential than necessary (the auth token)
          • the overall security of this is a bit questionable.
          1. Pass the credential from the "master" build downstream as a Password parameter.
            That's pretty straightforward, we extract the desired credential in the "master" build and pass it as a string downstream via a password parameter. (recently we had fun with JENKINS-62305, but that's another story).
            Downsides:
          • If the "master" build doesn't need the user credential, it has to access it anyways to pass it downstream.
          • Duplicate parameters for the same thing - credential for those who run the downstream job directly, password for the "master" job.
          • To at least hide this from the Build With Parameters screen, I've been omitting the password parameter declaration, but without it jenkins floods the logs with a security error about that it's being passed an undeclared parameter.

          Ideally, the downstream job would be able to access the user creds in both cases:

          • being triggered directly by the user
          • being triggered by an upstream build triggered by the user.

          If there's a way how to do that and have only one code / creds parameter for the downstream job and just passing the cred id from the upstream/"master" job, that would be a big relief. Right now, I don't see that option, but I'm not yet sure what the do next.

          You also say

          You can attach the actions directly via the low level API to simulate credential authorization by a user.

          Can you elaborate on that a bit, toss pointers to some docs or examples if that's still relevant?

          Thank you very much and sorry for the long post!

          Vít Zikmund added a comment - - edited Hi jvz , here's what we deal with. First of all, we do use the Authorize Project Plugin with all jobs set to: Run as the user who triggered the build. There's one "master" job with a credential parameter. Via that one, we can access the selected user-scope credential and all is nice and smooth. The problem is, that this job triggers another builds via the build() step and those builds need the "master" build credential as well. And that's where we hit the wall, or mud at least, as there's an ugly way through, but the credentials parameter is not a big help in either of the workarounds we've figured out: Use the credentials parameter in the downstream jobs and trigger them via Jenkins API and the user's auth token to pretend it was directly him/her who did it. This allows the authorize/credentials plugins to work properly and reach for the user's creds. The downside is that the triggering from the "master" job can't be done with the build() step, as that puts the master build as the main trigger cause, which IMHO kills the authorize plugin's Run as the user who triggered the build (even though the original user-triggered cause is also transitively visible) In one case we really reinvented the build() wheel to do that, and we made the users to save their auth tokens in their user creds and trigger the "master" build with it. For illustration the core looks like this: curl - sS -f -i -X POST "https://$JENKINS_HOSTNAME/job/$JOBNAME/buildWithParameters" --user "$USERNAME:$TOKEN"  -data-urlencode "user_cred=$DESIRED_USER_CRED_ID" Sad things: rewritten build() in "user space" user has to manage one more credential than necessary (the auth token) the overall security of this is a bit questionable. Pass the credential from the "master" build downstream as a Password parameter. That's pretty straightforward, we extract the desired credential in the "master" build and pass it as a string downstream via a password parameter. (recently we had fun with JENKINS-62305 , but that's another story). Downsides: If the "master" build doesn't need the user credential, it has to access it anyways to pass it downstream. Duplicate parameters for the same thing - credential for those who run the downstream job directly, password for the "master" job. To at least hide this from the Build With Parameters screen, I've been omitting the password parameter declaration, but without it jenkins floods the logs with a security error about that it's being passed an undeclared parameter. Ideally, the downstream job would be able to access the user creds in both cases: being triggered directly by the user being triggered by an upstream build triggered by the user. If there's a way how to do that and have only one code / creds parameter for the downstream job and just passing the cred id from the upstream/"master" job, that would be a big relief. Right now, I don't see that option, but I'm not yet sure what the do next. You also say You can attach the actions directly via the low level API to simulate credential authorization by a user. Can you elaborate on that a bit, toss pointers to some docs or examples if that's still relevant? Thank you very much and sorry for the long post!

          Matt Sicker added a comment -

          You've primarily described https://issues.jenkins.io/browse/JENKINS-59109 which would add support for propagating user credentials from one build to another via the build step. The low level API is described here: https://github.com/jenkinsci/credentials-plugin/blob/master/docs/consumer.adoc#binding-user-supplied-credentials-parameters-to-builds which would be used in an implementation for JENKINS-59109 and any other plugins that wish to integrate this further. Ideally, this feature would offer the ability to specify which loaded credentials to propagate. The gist of the integration would propagate the user who bound each credential parameter so that downstream jobs still know which user corresponds to which credential binding for authorization purposes.

          Matt Sicker added a comment - You've primarily described https://issues.jenkins.io/browse/JENKINS-59109 which would add support for propagating user credentials from one build to another via the build step. The low level API is described here: https://github.com/jenkinsci/credentials-plugin/blob/master/docs/consumer.adoc#binding-user-supplied-credentials-parameters-to-builds which would be used in an implementation for JENKINS-59109 and any other plugins that wish to integrate this further. Ideally, this feature would offer the ability to specify which loaded credentials to propagate. The gist of the integration would propagate the user who bound each credential parameter so that downstream jobs still know which user corresponds to which credential binding for authorization purposes.

            pskopek Peter Skopek
            cleclerc Cyrille Le Clerc
            Votes:
            21 Vote for this issue
            Watchers:
            26 Start watching this issue

              Created:
              Updated: