Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44772

User Scoped credentials are not used by the "withCredentials" pipeline step

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Environment

      • Jenkins 2.46.2
      • credentials:2.1.13 'Credentials Plugin'
      • credentials-binding:1.11 'Credentials Binding Plugin'
      • workflow-aggregator:2.5 'Pipeline'
      • workflow-api:2.12 'Pipeline: API'
      • workflow-basic-steps:2.4 'Pipeline: Basic Steps'
      • workflow-cps:2.29 'Pipeline: Groovy'
      • workflow-cps-checkpoint:2.4 'CloudBees Pipeline: Groovy Checkpoint Plugin'
      • workflow-cps-global-lib:2.7 'Pipeline: Shared Groovy Libraries'
      • workflow-durable-task-step:2.10 'Pipeline: Nodes and Processes'
      • workflow-job:2.10 'Pipeline: Job'
      • workflow-multibranch:2.14 'Pipeline: Multibranch'
      • workflow-scm-step:2.4 'Pipeline: SCM Step'
      • workflow-step-api:2.9 'Pipeline: Step API'
      • workflow-support:2.14 'Pipeline: Supporting APIs'

      Scenario to Use User Scoped Credentials in "withCredentials()"

      Many organizations use GPG Signing Key and special permissions on Nexus / Artifactory to create releases. For traceability and security, these privileged credentials may be managed as are "per individual/personal credentials", they may not be shared with team members.

      For this kind of credentials, we want to use Jenkins User Scoped Credentials in pipeline (withCredentials, git, config-file-provider, ssh-agent...)

      Description

      When using the authorize project plugin,

      • User Scoped Credentials are not found by the "withCredentials" pipeline step.
      • Global Credentials overwritten by user scoped credentials are not overwritten when used with the "withCredentials" pipeline step.

      Reproduce

      • Install the Project Authorize Plugin and configure it "Run as user who triggered the build"
      • Create a global credential "global-credentials-intended-to-be-overwritten-at-the-user-scope"
      • Create user scoped credentials "global-credentials-intended-to-be-overwritten-at-the-user-scope"
      • create a pipeline with "withCredentials" binding 'global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope' and writing it in a text file
      • run the build, open the text file in the workspace and verify that the global credentials are NOT overwritten
      • Create user scoped credentials "my-username-password"
      • Create a pipeline with "withCredentials" and the "my-username-password" credentials
      • job will fail with "CredentialNotFoundException"
      node {
          // verify that the build is properly impersonated by the https://wiki.jenkins-ci.org/display/JENKINS/Authorize+Project+plugin
          echo "Build is running as user " + org.acegisecurity.context.SecurityContextHolder.getContext().getAuthentication().toString()
          
          stage ("Global Credentials Overwritten at the user scope") {
              // credentials declared globally and overwritten by a user scoped credentials
              withCredentials([
                  usernamePassword(
                      credentialsId: 'global-credentials-intended-to-be-overwritten-at-the-user-scope', 
                      passwordVariable: 'PASSWORD_VAR', 
                      usernameVariable: 'USERNAME_VAR')]) {
                 sh "echo $PASSWORD_VAR > spy-overwritten-creds.txt"
              }
          }
          stage ("User Scoped Credentials") {
              withCredentials([
                  usernamePassword(
                      credentialsId: 'my-username-password', 
                      passwordVariable: 'PASSWORD_VAR', 
                      usernameVariable: 'USERNAME_VAR')]) {
                 sh "echo $PASSWORD_VAR > spy-user-scoped-credentials.txt"
              }
          }
      }
      
      Started by user admin
      [Pipeline] node
      Running on agent-1 in /home/ubuntu/jenkins-aws-home/workspace/tests/user-scoped-credentials-pipeline-step-with-credentials
      [Pipeline] {
      [Pipeline] echo
      Build is running as user org.acegisecurity.providers.UsernamePasswordAuthenticationToken@965748a4: Username: admin; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: authenticated
      [Pipeline] stage
      [Pipeline] { (Global Credentials Overwritten at the user scope)
      [Pipeline] withCredentials
      [Pipeline] {
      [Pipeline] sh
      [user-scoped-credentials-pipeline-step-with-credentials] Running shell script
      + echo ****
      [Pipeline] }
      [Pipeline] // withCredentials
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (User Scoped Credentials)
      [Pipeline] withCredentials
      [Pipeline] // withCredentials
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      org.jenkinsci.plugins.credentialsbinding.impl.CredentialNotFoundException: my-username-password
        at org.jenkinsci.plugins.credentialsbinding.MultiBinding.getCredentials(MultiBinding.java:153)
        at org.jenkinsci.plugins.credentialsbinding.impl.UsernamePasswordMultiBinding.bind(UsernamePasswordMultiBinding.java:76)
        at org.jenkinsci.plugins.credentialsbinding.impl.BindingStep$Execution.start(BindingStep.java:114)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:184)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:126)
        at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
        at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
        at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151)
        at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:21)
        at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:115)
        at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
        at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
        at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16)
        at WorkflowScript.run(WorkflowScript:16)
        at ___cps.transform___(Native Method)
        at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
        at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
        at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
        at sun.reflect.GeneratedMethodAccessor591.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
        at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
        at com.cloudbees.groovy.cps.Next.step(Next.java:74)
        at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
        at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
        at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
        at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:165)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:328)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228)
        at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
        at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
      Finished: FAILURE
      

        Attachments

          Issue Links

            Activity

            Hide
            vit_zikmund Vít Zikmund added a comment -

            Hello Matt Sicker, I'm quite sure it is not the case. Your JENIKINS-58170 is nice, but it doesn't allow you to access the userscope credentials without them being supplied in a parameter.
            I suppose Cyrille Le Clerc was trying to avoid using the build parameter entirely.

            I personally don't see the security gain in this, especially if it's me, who's running the job, why wouldn't I want the job to access my user credentials? I know there's a possibility that if this got enabled, the job could guess and snoop any user's credential, but there could also be a way for the job to declare which credentials it's about to be accessing which the user would (until further change) accept only once and use it without selecting a credentials parameter each time s/he runs the build.

            I actually found a way how to circumvent the limitation by creating an upstream job, that uses a user's jenkins API token to trigger the job with credentials via the Jenkins API (so the downstream build thinks it was directly run by the user, which is one of the requirements for the user creds to be read), but it required me/my peers to reinvent the whole build() logic with curl, and that just sucked. I'd still give an eternal praise to someone willing to reconsider these limitations to make them more user friendly.

            Show
            vit_zikmund Vít Zikmund added a comment - Hello Matt Sicker , I'm quite sure it is not the case. Your JENIKINS-58170 is nice, but it doesn't allow you to access the userscope credentials without them being supplied in a parameter. I suppose Cyrille Le Clerc was trying to avoid using the build parameter entirely. I personally don't see the security gain in this, especially if it's me, who's running the job, why wouldn't I want the job to access my user credentials? I know there's a possibility that if this got enabled, the job could guess and snoop any user's credential, but there could also be a way for the job to declare which credentials it's about to be accessing which the user would (until further change) accept only once and use it without selecting a credentials parameter each time s/he runs the build. I actually found a way how to circumvent the limitation by creating an upstream job, that uses a user's jenkins API token to trigger the job with credentials via the Jenkins API (so the downstream build thinks it was directly run by the user, which is one of the requirements for the user creds to be read), but it required me/my peers to reinvent the whole build() logic with curl, and that just sucked. I'd still give an eternal praise to someone willing to reconsider these limitations to make them more user friendly.
            Hide
            jvz Matt Sicker added a comment -

            The parameter API is the high level API. You can attach the actions directly via the low level API to simulate credential authorization by a user.

            Show
            jvz Matt Sicker added a comment - The parameter API is the high level API. You can attach the actions directly via the low level API to simulate credential authorization by a user.
            Hide
            jvz Matt Sicker added a comment -

            Vít Zikmund: are you using authorize project? If the build is running as the SYSTEM user, then it won't have access to other user's credentials as it's, you know, a different user.

            Show
            jvz Matt Sicker added a comment - Vít Zikmund : are you using authorize project? If the build is running as the SYSTEM user, then it won't have access to other user's credentials as it's, you know, a different user.
            Hide
            vit_zikmund Vít Zikmund added a comment - - edited

            Hi Matt Sicker, here's what we deal with. First of all, we do use the Authorize Project Plugin with all jobs set to: Run as the user who triggered the build.

            There's one "master" job with a credential parameter. Via that one, we can access the selected user-scope credential and all is nice and smooth.

            The problem is, that this job triggers another builds via the build() step and those builds need the "master" build credential as well. And that's where we hit the wall, or mud at least, as there's an ugly way through, but the credentials parameter is not a big help in either of the workarounds we've figured out:

            1. Use the credentials parameter in the downstream jobs and trigger them via Jenkins API and the user's auth token to pretend it was directly him/her who did it.
              This allows the authorize/credentials plugins to work properly and reach for the user's creds.
              The downside is that the triggering from the "master" job can't be done with the build() step, as that puts the master build as the main trigger cause, which IMHO kills the authorize plugin's Run as the user who triggered the build (even though the original user-triggered cause is also transitively visible)
              In one case we really reinvented the build() wheel to do that, and we made the users to save their auth tokens in their user creds and trigger the "master" build with it.
              For illustration the core looks like this:
              curl -sS -f -i -X POST "https://$JENKINS_HOSTNAME/job/$JOBNAME/buildWithParameters" --user "$USERNAME:$TOKEN" -data-urlencode "user_cred=$DESIRED_USER_CRED_ID"
              Sad things:
            • rewritten build() in "user space"
            • user has to manage one more credential than necessary (the auth token)
            • the overall security of this is a bit questionable.
            1. Pass the credential from the "master" build downstream as a Password parameter.
              That's pretty straightforward, we extract the desired credential in the "master" build and pass it as a string downstream via a password parameter. (recently we had fun with JENKINS-62305, but that's another story).
              Downsides:
            • If the "master" build doesn't need the user credential, it has to access it anyways to pass it downstream.
            • Duplicate parameters for the same thing - credential for those who run the downstream job directly, password for the "master" job.
            • To at least hide this from the Build With Parameters screen, I've been omitting the password parameter declaration, but without it jenkins floods the logs with a security error about that it's being passed an undeclared parameter.

            Ideally, the downstream job would be able to access the user creds in both cases:

            • being triggered directly by the user
            • being triggered by an upstream build triggered by the user.

            If there's a way how to do that and have only one code / creds parameter for the downstream job and just passing the cred id from the upstream/"master" job, that would be a big relief. Right now, I don't see that option, but I'm not yet sure what the do next.

            You also say

            You can attach the actions directly via the low level API to simulate credential authorization by a user.

            Can you elaborate on that a bit, toss pointers to some docs or examples if that's still relevant?

            Thank you very much and sorry for the long post!

            Show
            vit_zikmund Vít Zikmund added a comment - - edited Hi Matt Sicker , here's what we deal with. First of all, we do use the Authorize Project Plugin with all jobs set to: Run as the user who triggered the build. There's one "master" job with a credential parameter. Via that one, we can access the selected user-scope credential and all is nice and smooth. The problem is, that this job triggers another builds via the build() step and those builds need the "master" build credential as well. And that's where we hit the wall, or mud at least, as there's an ugly way through, but the credentials parameter is not a big help in either of the workarounds we've figured out: Use the credentials parameter in the downstream jobs and trigger them via Jenkins API and the user's auth token to pretend it was directly him/her who did it. This allows the authorize/credentials plugins to work properly and reach for the user's creds. The downside is that the triggering from the "master" job can't be done with the build() step, as that puts the master build as the main trigger cause, which IMHO kills the authorize plugin's Run as the user who triggered the build (even though the original user-triggered cause is also transitively visible) In one case we really reinvented the build() wheel to do that, and we made the users to save their auth tokens in their user creds and trigger the "master" build with it. For illustration the core looks like this: curl - sS -f -i -X POST "https://$JENKINS_HOSTNAME/job/$JOBNAME/buildWithParameters" --user "$USERNAME:$TOKEN"  -data-urlencode "user_cred=$DESIRED_USER_CRED_ID" Sad things: rewritten build() in "user space" user has to manage one more credential than necessary (the auth token) the overall security of this is a bit questionable. Pass the credential from the "master" build downstream as a Password parameter. That's pretty straightforward, we extract the desired credential in the "master" build and pass it as a string downstream via a password parameter. (recently we had fun with JENKINS-62305 , but that's another story). Downsides: If the "master" build doesn't need the user credential, it has to access it anyways to pass it downstream. Duplicate parameters for the same thing - credential for those who run the downstream job directly, password for the "master" job. To at least hide this from the Build With Parameters screen, I've been omitting the password parameter declaration, but without it jenkins floods the logs with a security error about that it's being passed an undeclared parameter. Ideally, the downstream job would be able to access the user creds in both cases: being triggered directly by the user being triggered by an upstream build triggered by the user. If there's a way how to do that and have only one code / creds parameter for the downstream job and just passing the cred id from the upstream/"master" job, that would be a big relief. Right now, I don't see that option, but I'm not yet sure what the do next. You also say You can attach the actions directly via the low level API to simulate credential authorization by a user. Can you elaborate on that a bit, toss pointers to some docs or examples if that's still relevant? Thank you very much and sorry for the long post!
            Hide
            jvz Matt Sicker added a comment -

            You've primarily described https://issues.jenkins.io/browse/JENKINS-59109 which would add support for propagating user credentials from one build to another via the build step. The low level API is described here: https://github.com/jenkinsci/credentials-plugin/blob/master/docs/consumer.adoc#binding-user-supplied-credentials-parameters-to-builds which would be used in an implementation for JENKINS-59109 and any other plugins that wish to integrate this further. Ideally, this feature would offer the ability to specify which loaded credentials to propagate. The gist of the integration would propagate the user who bound each credential parameter so that downstream jobs still know which user corresponds to which credential binding for authorization purposes.

            Show
            jvz Matt Sicker added a comment - You've primarily described https://issues.jenkins.io/browse/JENKINS-59109 which would add support for propagating user credentials from one build to another via the build step. The low level API is described here: https://github.com/jenkinsci/credentials-plugin/blob/master/docs/consumer.adoc#binding-user-supplied-credentials-parameters-to-builds which would be used in an implementation for JENKINS-59109 and any other plugins that wish to integrate this further. Ideally, this feature would offer the ability to specify which loaded credentials to propagate. The gist of the integration would propagate the user who bound each credential parameter so that downstream jobs still know which user corresponds to which credential binding for authorization purposes.

              People

              Assignee:
              pskopek Peter Skopek
              Reporter:
              cleclerc Cyrille Le Clerc
              Votes:
              19 Vote for this issue
              Watchers:
              24 Start watching this issue

                Dates

                Created:
                Updated: