Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44773

User Scoped credentials are not used by the "git" pipeline step

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Environment

      • Jenkins 2.46.2
      • credentials:2.1.13 'Credentials Plugin'
      • workflow-aggregator:2.5 'Pipeline'
      • workflow-api:2.12 'Pipeline: API'
      • workflow-basic-steps:2.4 'Pipeline: Basic Steps'
      • workflow-cps:2.29 'Pipeline: Groovy'
      • workflow-cps-checkpoint:2.4 'CloudBees Pipeline: Groovy Checkpoint Plugin'
      • workflow-cps-global-lib:2.7 'Pipeline: Shared Groovy Libraries'
      • workflow-durable-task-step:2.10 'Pipeline: Nodes and Processes'
      • workflow-job:2.10 'Pipeline: Job'
      • workflow-multibranch:2.14 'Pipeline: Multibranch'
      • workflow-scm-step:2.4 'Pipeline: SCM Step'
      • workflow-step-api:2.9 'Pipeline: Step API'
      • workflow-support:2.14 'Pipeline: Supporting APIs'

      Description

      When using the authorize project plugin,

      • User Scoped Credentials are not found by the "git" pipeline step.
      • Global Credentials overwritten by user scoped credentials are not overwritten by the "git" pipeline step.

      Reproduce

      • Install the Project Authorize Plugin and configure it "Run as user who triggered the build"
      • Create a global credential "global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope" with valid credentials for a bitbucket server
      • Create user scoped credentials "global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope" with invalid username and invalid password
      • create a pipeline with "git credentialsId: 'global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope', url: 'https://bitbucket.example.com/.....git'"
      • run the build, it will succeed and logs will show that the global creds have been used, a log message will show the valid username, proving that the user credentials have been ignored
      • Create user scoped credentials "user-scoped-bitbucket-creds" with valid credentials for a bitbucket server
      • Create a pipeline with "git credentialsId: 'user-scoped-bitbucket-cred', url: 'https://bitbucket.example.com/.....git'"
      • job will fail with "authentication failed"

       

      node {
          // verify that the build is properly impersonated by the https://wiki.jenkins-ci.org/display/JENKINS/Authorize+Project+plugin
          echo "Build is running as user " + org.acegisecurity.context.SecurityContextHolder.getContext().getAuthentication().toString()
          
          stage ("Global Credentials Overwritten at the user scope") {
              // credentials declared globally and overwritten by a user scoped credentials
              git credentialsId: 'global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope', url: 'https://bitbucket.example.com/scm/pet/spring-petclinic.git'
          }
          
          stage ("User Scoped Credentials") {
              // user scoped credentials
              git credentialsId: 'user-scoped-bitbucket-creds', url: 'https://bitbucket.example.com/scm/pet/spring-petclinic.git'
          }
      }
      
      Started by user admin
      [Pipeline] node
      Running on agent-1 in /home/ubuntu/agent-home/workspace/tests/user-scoped-credentials-pipeline-step-git
      [Pipeline] {
      [Pipeline] echo
      Build is running as user org.acegisecurity.providers.UsernamePasswordAuthenticationToken@965748a4: Username: admin; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: authenticated
      [Pipeline] stage
      [Pipeline] { (Global Credentials Overwritten at the user scope)
      [Pipeline] git
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
       > git config remote.origin.url https://bitbucket.example.com/scm/pet/spring-petclinic.git # timeout=10
      Fetching upstream changes from https://bitbucket.example.com/scm/pet/spring-petclinic.git
       > git --version # timeout=10
      using GIT_ASKPASS to set credentials QAh4dAzetrEp global creds, not overwritten
       > git fetch --tags --progress https://bitbucket.example.com/scm/pet/spring-petclinic.git +refs/heads/*:refs/remotes/origin/*
       > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
       > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
      Checking out Revision fd1c742d4f8d193eb935519909c15302b783cd52 (refs/remotes/origin/master)
       > git config core.sparsecheckout # timeout=10
       > git checkout -f fd1c742d4f8d193eb935519909c15302b783cd52
       > git branch -a -v --no-abbrev # timeout=10
       > git branch -D master # timeout=10
       > git checkout -b master fd1c742d4f8d193eb935519909c15302b783cd52
       > git rev-list fd1c742d4f8d193eb935519909c15302b783cd52 # timeout=10
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (User Scoped Credentials)
      [Pipeline] git
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
       > git config remote.origin.url https://bitbucket.example.com/scm/pet/spring-petclinic.git # timeout=10
      Fetching upstream changes from https://bitbucket.example.com/scm/pet/spring-petclinic.git
       > git --version # timeout=10
       > git fetch --tags --progress https://bitbucket.example.com/scm/pet/spring-petclinic.git +refs/heads/*:refs/remotes/origin/*
      ERROR: Error fetching remote repo 'origin'
      hudson.plugins.git.GitException: Failed to fetch from https://bitbucket.example.com/scm/pet/spring-petclinic.git
      	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:806)
      	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1070)
      	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1101)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:260)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://bitbucket.example.com/scm/pet/spring-petclinic.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
      stdout: 
      stderr: fatal: Authentication failed for 'https://bitbucket.example.com/scm/pet/spring-petclinic.git/'
      
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1793)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1519)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:64)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:315)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:153)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
      	at hudson.remoting.Request$2.run(Request.java:336)
      	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      	at ......remote call to agent-1(Native Method)
      	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
      	at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
      	at hudson.remoting.Channel.call(Channel.java:830)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
      	at sun.reflect.GeneratedMethodAccessor769.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
      	at com.sun.proxy.$Proxy120.execute(Unknown Source)
      	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:804)
      	... 13 more
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: null
      Finished: FAILURE
      

        Attachments

          Issue Links

            Activity

            Hide
            markewaite Mark Waite added a comment -

            I am not trying to fix this at this time Hugo van den Brand.

            Show
            markewaite Mark Waite added a comment - I am not trying to fix this at this time Hugo van den Brand .
            Hide
            jvz Matt Sicker added a comment -

            This feature can be replaced by JENKINS-58170 which provides a more standardized approach to granting user-scoped credentials to builds.

            Show
            jvz Matt Sicker added a comment - This feature can be replaced by JENKINS-58170 which provides a more standardized approach to granting user-scoped credentials to builds.
            Hide
            jvz Matt Sicker added a comment -

            There is now a new way to access user-scoped credentials detailed in JENKINS-58170 which uses credentials build parameters. These allow user-scoped credentials to be referenced in any context where you'd normally use the credentials id.

            Show
            jvz Matt Sicker added a comment - There is now a new way to access user-scoped credentials detailed in JENKINS-58170 which uses credentials build parameters. These allow user-scoped credentials to be referenced in any context where you'd normally use the credentials id.
            Hide
            mdlugaj Marek Dlugajczyk added a comment - - edited

            So I have a question. We are using the checkout step in our pipelines to clone, get all the metadata about repo and see the details about repositories in Jenkins GUI. Also checkout step is the recommended to use.

            In our case we have Pipelines that have credentials build parameter of type UsernamePassword (as we clone repos over https) and we require those credentials to be user scoped. Unfortunately when we try to use user scoped credentials with the checkout step it can't clone our private repositories from Gitlab.

            I've tried to use the credentials ID that comes from the build parameter and also tried the build parameter name instead of ID as described in https://issues.jenkins.io/browse/JENKINS-58170 however this does not work as in case of the withCredentials block. To be clear, with the withCredentials block I had no issues to use user scoped credentials.

             

            Consider we have credentials build parameter of type UsernamePassword named in GUI: RepoCredentials - these must be/use user scoped credentials in the build.

            Case 1 - we use the ID of the credentials by passing the value of parameter selected from GUI to checkout step:

            (...)
            script {
                scmVars = checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [
                    [name: params.RepoRef]
                ], doGenerateSubmoduleConfigurations: false, extensions: [
                    [$class: 'WipeWorkspace'],
                    [$class: 'AuthorInChangelog'],
                ], submoduleCfg: [], userRemoteConfigs: [
                    [url: params.RepoUrl, credentialsId: params.RepoCredentials]
                ]]
            }
            (...)

            This does not work properly and we get: 

            [Pipeline] checkout
             Wiping out workspace first.
             Cloning the remote Git repository
             Cloning repository https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git
              > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10
             Fetching upstream changes from https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git
              > git --version # timeout=10
              > git --version # 'git version 2.17.1'
              > git fetch --tags --progress -- https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10
             ERROR: Error cloning remote repo 'origin'
             hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
             stdout: 
             stderr: remote: HTTP Basic: Access denied
             fatal: Authentication failed for 'https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git/'
             
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799)
                 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
                 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
                 at hudson.remoting.UserRequest.perform(UserRequest.java:211)
                 at hudson.remoting.UserRequest.perform(UserRequest.java:54)
                 at hudson.remoting.Request$2.run(Request.java:376)
                 at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
                 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                 at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
                 at java.lang.Thread.run(Thread.java:748)
                 Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:49712
                     at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
                     at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
                     at hudson.remoting.Channel.call(Channel.java:1001)
                     at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
                     at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source)
                     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                     at java.lang.reflect.Method.invoke(Method.java:498)
                     at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
                     at com.sun.proxy.$Proxy393.execute(Unknown Source)
                     at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222)
                     at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300)
                     at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125)
                     at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93)
                     at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80)
                     at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
                     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                     ... 1 more

            (I replaced Gitlab URL and repo name for example values)

             

            Case 2 - we use the name of the credentials parameter by passing the name to checkout step as suggested in https://issues.jenkins.io/browse/JENKINS-58170:

            (...)
            script {
                scmVars = checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [
                    [name: params.RepoRef]
                ], doGenerateSubmoduleConfigurations: false, extensions: [
                    [$class: 'WipeWorkspace'],
                    [$class: 'AuthorInChangelog'],
                ], submoduleCfg: [], userRemoteConfigs: [
                    [url: params.RepoUrl, credentialsId: 'RepoCredentials']
                ]]
            }
            (...)
            

            But again that does not work that way and we get:

            [Pipeline] checkout
             Wiping out workspace first.
             Cloning the remote Git repository
             Cloning repository https://githubhost.com/cicd/ansible/playbooks/example-repository.git
              > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10
             Fetching upstream changes from https://githubhost.com/cicd/ansible/playbooks/example-repository.git
              > git --version # timeout=10
              > git --version # 'git version 2.17.1'
              > git fetch --tags --progress -- https://githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10
             ERROR: Error cloning remote repo 'origin'
             hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https://githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
             stdout: 
             stderr: remote: HTTP Basic: Access denied
             fatal: Authentication failed for 'https://githubhost.com/cicd/ansible/playbooks/example-repository.git/'
             
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570)
                 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799)
                 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
                 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
                 at hudson.remoting.UserRequest.perform(UserRequest.java:211)
                 at hudson.remoting.UserRequest.perform(UserRequest.java:54)
                 at hudson.remoting.Request$2.run(Request.java:376)
                 at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
                 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                 at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
                 at java.lang.Thread.run(Thread.java:748)
                 Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:42646
                     at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
                     at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
                     at hudson.remoting.Channel.call(Channel.java:1001)
                     at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
                     at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source)
                     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                     at java.lang.reflect.Method.invoke(Method.java:498)
                     at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
                     at com.sun.proxy.$Proxy393.execute(Unknown Source)
                     at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222)
                     at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300)
                     at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125)
                     at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93)
                     at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80)
                     at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
                     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                     ... 1 more

            (btw. I double checked the credentials configured in Jenkins to make sure there are no typos etc. as we got: "returned status code 128", "HTTP Basic: Access denied")

             

            So in general my question is how we can use user scoped credentials with the checkout step the correct way? Matt Sicker you wrote that it is possible to access user scoped credentials in: "any context where you'd normally use the credentials id." Could you explain what I'm missing here?

             

            I also want to add that I've found a workaround for this but it's not pretty IMHO. And I'm really curious how it should be done correctly. I'll post the workaround in separate comment as it might be useful.

            Show
            mdlugaj Marek Dlugajczyk added a comment - - edited So I have a question. We are using the checkout step in our pipelines to clone, get all the metadata about repo and see the details about repositories in Jenkins GUI. Also checkout step is the recommended to use. In our case we have Pipelines that have credentials build parameter of type UsernamePassword (as we clone repos over https) and we require those credentials to be user scoped . Unfortunately when we try to use user scoped credentials with the checkout step it can't clone our private repositories from Gitlab. I've tried to use the credentials ID that comes from the build parameter and also tried the build parameter name instead of ID as described in https://issues.jenkins.io/browse/JENKINS-58170  however this does not work as in case of the withCredentials block. To be clear, with the withCredentials block I had no issues to use user scoped credentials.   Consider we have credentials build parameter of type UsernamePassword named in GUI: RepoCredentials - these must be/use user scoped credentials in the build. Case 1 - we use the ID of the credentials by passing the value of parameter selected from GUI to checkout step: (...) script { scmVars = checkout changelog: true , poll: true , scm: [$class: 'GitSCM' , branches: [ [name: params.RepoRef] ], doGenerateSubmoduleConfigurations: false , extensions: [ [$class: 'WipeWorkspace' ], [$class: 'AuthorInChangelog' ], ], submoduleCfg: [], userRemoteConfigs: [ [url: params.RepoUrl, credentialsId: params.RepoCredentials] ]] } (...) This does not work properly and we get:  [Pipeline] checkout Wiping out workspace first. Cloning the remote Git repository Cloning repository https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10 Fetching upstream changes from https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git > git --version # timeout=10 > git --version # 'git version 2.17.1' > git fetch --tags --progress -- https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10 ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: remote: HTTP Basic: Access denied fatal: Authentication failed for 'https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git/' at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:376) at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119) at java.lang. Thread .run( Thread .java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:49712 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy393.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more (I replaced Gitlab URL and repo name for example values)   Case 2 - we use the name of the credentials parameter by passing the name to checkout step as suggested in https://issues.jenkins.io/browse/JENKINS-58170: (...) script { scmVars = checkout changelog: true , poll: true , scm: [$class: 'GitSCM' , branches: [ [name: params.RepoRef] ], doGenerateSubmoduleConfigurations: false , extensions: [ [$class: 'WipeWorkspace' ], [$class: 'AuthorInChangelog' ], ], submoduleCfg: [], userRemoteConfigs: [ [url: params.RepoUrl, credentialsId: 'RepoCredentials' ] ]] } (...) But again that does not work that way and we get: [Pipeline] checkout Wiping out workspace first. Cloning the remote Git repository Cloning repository https: //githubhost.com/cicd/ansible/playbooks/example-repository.git > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10 Fetching upstream changes from https: //githubhost.com/cicd/ansible/playbooks/example-repository.git > git --version # timeout=10 > git --version # 'git version 2.17.1' > git fetch --tags --progress -- https: //githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10 ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https: //githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: remote: HTTP Basic: Access denied fatal: Authentication failed for 'https: //githubhost.com/cicd/ansible/playbooks/example-repository.git/' at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:376) at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119) at java.lang. Thread .run( Thread .java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:42646 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy393.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more (btw. I double checked the credentials configured in Jenkins to make sure there are no typos etc. as we got: "returned status code 128", "HTTP Basic: Access denied" )   So in general my question is how we can use user scoped credentials with the checkout step the correct way? Matt Sicker  you wrote that it is possible to access user scoped credentials in: "any context where you'd normally use the credentials id." Could you explain what I'm missing here?   I also want to add that I've found a workaround for this but it's not pretty IMHO. And I'm really curious how it should be done correctly. I'll post the workaround in separate comment as it might be useful.
            Hide
            mdlugaj Marek Dlugajczyk added a comment - - edited

            As mentioned above I'm posting my "ugly" workaround for using user scoped credentials with checkout step:

             

            Consider we have credentials build parameter of type UsernamePassword named in GUI: RepoCredentials - these must be/use user scoped credentials in the build.

            (...)
            stage('Clone') {
                steps {
                    retry(3) {
                        script {
                            /**
                             * All steps below are temporary workaround for user scoped credentials not being resolved by the
                             * checkout/git steps, see https://issues.jenkins.io/browse/JENKINS-44773
                             * Once this will work properly in Jenkins we can just do simple checkout step as normal, until
                             * that time we need to use withCredentials as this can resolve user scoped credentials without a problem
                             * Tried using GIT_ASKPASS method here as well but that didn't work at all in checkout step
                             * We still want to use the checkout step for repo details in Jenkins GUI that this step sets
                             */
                            withCredentials([
                                usernamePassword(credentialsId: 'RepoCredentials', usernameVariable: 'GIT_USR', passwordVariable: 'GIT_PSW')
                            ]) {                    
            
                                def tmpGitPassFilePath = "${pwd(tmp: true)}/.git-credentials"
                                URL projectURL = new URL(params.RepoUrl)                    
                                
                                /**
                                 * Using writeFile as echoing this in 'sh' step would print contents in BlueOcean
                                 * We need to encode Git password if it has special characters, encoded spaces '+' sign
                                 * must be converted to '%20'
                                 */
                                def encodedPsw = URLEncoder.encode(GIT_PSW, 'UTF-8').replace('+', '%20')
                                writeFile file: tmpGitPassFilePath, text: """\
                                    ${projectURL.getProtocol()}://${GIT_USR}:${encodedPsw}@${projectURL.getHost()}
                                    """.stripIndent()                    
            
                                sh label: 'Configure local Git before checkout', script: """
                                    #!/bin/bash +x
                                    git config --global credential.helper 'store --file=${tmpGitPassFilePath}'
                                    """.stripIndent().trim()                    
                                
                                scmVars = checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [
                                    [name: params.RepoRef]
                                ], doGenerateSubmoduleConfigurations: false, extensions: [
                                    [$class: 'WipeWorkspace'],
                                    [$class: 'AuthorInChangelog'],
                                ], submoduleCfg: [], userRemoteConfigs: [
                                    /**
                                     * No credentialsID provided to use our local git temp. config
                                     * see https://github.com/jenkinsci/git-plugin/blob/de97701cb452c20f967072a1cab99ad2efb96425/src/main/java/jenkins/plugins/git/GitSCMBuilder.java#L116
                                     */
                                    [url: params.RepoUrl]
                                ]]                    
            
                                sh label: 'Clear temporary Git config after checkout', script: """
                                    #!/bin/bash +x
                                    git config --global --unset credential.helper
                                    rm -f ${tmpGitPassFilePath}
                                    """.stripIndent().trim()
                            }
                        }
                    }
                }
            }
            Show
            mdlugaj Marek Dlugajczyk added a comment - - edited As mentioned above I'm posting my "ugly" workaround for using user scoped credentials with checkout step:   Consider we have credentials build parameter of type UsernamePassword named in GUI:  RepoCredentials  - these must be/use user scoped credentials in the build. (...) stage( 'Clone' ) { steps { retry(3) { script { /** * All steps below are temporary workaround for user scoped credentials not being resolved by the * checkout/git steps, see https: //issues.jenkins.io/browse/JENKINS-44773 * Once this will work properly in Jenkins we can just do simple checkout step as normal, until * that time we need to use withCredentials as this can resolve user scoped credentials without a problem * Tried using GIT_ASKPASS method here as well but that didn't work at all in checkout step * We still want to use the checkout step for repo details in Jenkins GUI that this step sets */ withCredentials([ usernamePassword(credentialsId: 'RepoCredentials' , usernameVariable: 'GIT_USR' , passwordVariable: 'GIT_PSW' ) ]) { def tmpGitPassFilePath = "${pwd(tmp: true )}/.git-credentials" URL projectURL = new URL(params.RepoUrl) /** * Using writeFile as echoing this in 'sh' step would print contents in BlueOcean * We need to encode Git password if it has special characters, encoded spaces '+' sign * must be converted to '%20' */ def encodedPsw = URLEncoder.encode(GIT_PSW, 'UTF-8' ).replace( '+' , '%20' ) writeFile file: tmpGitPassFilePath, text: """\ ${projectURL.getProtocol()}: //${GIT_USR}:${encodedPsw}@${projectURL.getHost()} """.stripIndent() sh label: 'Configure local Git before checkout' , script: """ #!/bin/bash +x git config --global credential.helper 'store --file=${tmpGitPassFilePath}' """.stripIndent().trim() scmVars = checkout changelog: true , poll: true , scm: [$class: 'GitSCM' , branches: [ [name: params.RepoRef] ], doGenerateSubmoduleConfigurations: false , extensions: [ [$class: 'WipeWorkspace' ], [$class: 'AuthorInChangelog' ], ], submoduleCfg: [], userRemoteConfigs: [ /** * No credentialsID provided to use our local git temp. config * see https: //github.com/jenkinsci/git-plugin/blob/de97701cb452c20f967072a1cab99ad2efb96425/src/main/java/jenkins/plugins/git/GitSCMBuilder.java#L116 */ [url: params.RepoUrl] ]] sh label: 'Clear temporary Git config after checkout' , script: """ #!/bin/bash +x git config --global --unset credential.helper rm -f ${tmpGitPassFilePath} """.stripIndent().trim() } } } } }

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              cleclerc Cyrille Le Clerc
              Votes:
              6 Vote for this issue
              Watchers:
              13 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: