Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44773

User Scoped credentials are not used by the "git" pipeline step

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Major Major
    • git-plugin

      Environment

      • Jenkins 2.46.2
      • credentials:2.1.13 'Credentials Plugin'
      • workflow-aggregator:2.5 'Pipeline'
      • workflow-api:2.12 'Pipeline: API'
      • workflow-basic-steps:2.4 'Pipeline: Basic Steps'
      • workflow-cps:2.29 'Pipeline: Groovy'
      • workflow-cps-checkpoint:2.4 'CloudBees Pipeline: Groovy Checkpoint Plugin'
      • workflow-cps-global-lib:2.7 'Pipeline: Shared Groovy Libraries'
      • workflow-durable-task-step:2.10 'Pipeline: Nodes and Processes'
      • workflow-job:2.10 'Pipeline: Job'
      • workflow-multibranch:2.14 'Pipeline: Multibranch'
      • workflow-scm-step:2.4 'Pipeline: SCM Step'
      • workflow-step-api:2.9 'Pipeline: Step API'
      • workflow-support:2.14 'Pipeline: Supporting APIs'

      Description

      When using the authorize project plugin,

      • User Scoped Credentials are not found by the "git" pipeline step.
      • Global Credentials overwritten by user scoped credentials are not overwritten by the "git" pipeline step.

      Reproduce

      • Install the Project Authorize Plugin and configure it "Run as user who triggered the build"
      • Create a global credential "global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope" with valid credentials for a bitbucket server
      • Create user scoped credentials "global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope" with invalid username and invalid password
      • create a pipeline with "git credentialsId: 'global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope', url: 'https://bitbucket.example.com/.....git'"
      • run the build, it will succeed and logs will show that the global creds have been used, a log message will show the valid username, proving that the user credentials have been ignored
      • Create user scoped credentials "user-scoped-bitbucket-creds" with valid credentials for a bitbucket server
      • Create a pipeline with "git credentialsId: 'user-scoped-bitbucket-cred', url: 'https://bitbucket.example.com/.....git'"
      • job will fail with "authentication failed"

       

      node {
          // verify that the build is properly impersonated by the https://wiki.jenkins-ci.org/display/JENKINS/Authorize+Project+plugin
          echo "Build is running as user " + org.acegisecurity.context.SecurityContextHolder.getContext().getAuthentication().toString()
          
          stage ("Global Credentials Overwritten at the user scope") {
              // credentials declared globally and overwritten by a user scoped credentials
              git credentialsId: 'global-bitbucket-credentials-intended-to-be-overwritten-at-the-user-scope', url: 'https://bitbucket.example.com/scm/pet/spring-petclinic.git'
          }
          
          stage ("User Scoped Credentials") {
              // user scoped credentials
              git credentialsId: 'user-scoped-bitbucket-creds', url: 'https://bitbucket.example.com/scm/pet/spring-petclinic.git'
          }
      }
      
      Started by user admin
      [Pipeline] node
      Running on agent-1 in /home/ubuntu/agent-home/workspace/tests/user-scoped-credentials-pipeline-step-git
      [Pipeline] {
      [Pipeline] echo
      Build is running as user org.acegisecurity.providers.UsernamePasswordAuthenticationToken@965748a4: Username: admin; Password: [PROTECTED]; Authenticated: true; Details: null; Granted Authorities: authenticated
      [Pipeline] stage
      [Pipeline] { (Global Credentials Overwritten at the user scope)
      [Pipeline] git
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
       > git config remote.origin.url https://bitbucket.example.com/scm/pet/spring-petclinic.git # timeout=10
      Fetching upstream changes from https://bitbucket.example.com/scm/pet/spring-petclinic.git
       > git --version # timeout=10
      using GIT_ASKPASS to set credentials QAh4dAzetrEp global creds, not overwritten
       > git fetch --tags --progress https://bitbucket.example.com/scm/pet/spring-petclinic.git +refs/heads/*:refs/remotes/origin/*
       > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
       > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
      Checking out Revision fd1c742d4f8d193eb935519909c15302b783cd52 (refs/remotes/origin/master)
       > git config core.sparsecheckout # timeout=10
       > git checkout -f fd1c742d4f8d193eb935519909c15302b783cd52
       > git branch -a -v --no-abbrev # timeout=10
       > git branch -D master # timeout=10
       > git checkout -b master fd1c742d4f8d193eb935519909c15302b783cd52
       > git rev-list fd1c742d4f8d193eb935519909c15302b783cd52 # timeout=10
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (User Scoped Credentials)
      [Pipeline] git
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
       > git config remote.origin.url https://bitbucket.example.com/scm/pet/spring-petclinic.git # timeout=10
      Fetching upstream changes from https://bitbucket.example.com/scm/pet/spring-petclinic.git
       > git --version # timeout=10
       > git fetch --tags --progress https://bitbucket.example.com/scm/pet/spring-petclinic.git +refs/heads/*:refs/remotes/origin/*
      ERROR: Error fetching remote repo 'origin'
      hudson.plugins.git.GitException: Failed to fetch from https://bitbucket.example.com/scm/pet/spring-petclinic.git
      	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:806)
      	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1070)
      	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1101)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:260)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://bitbucket.example.com/scm/pet/spring-petclinic.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
      stdout: 
      stderr: fatal: Authentication failed for 'https://bitbucket.example.com/scm/pet/spring-petclinic.git/'
      
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1793)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1519)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:64)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:315)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:153)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
      	at hudson.remoting.Request$2.run(Request.java:336)
      	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      	at ......remote call to agent-1(Native Method)
      	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
      	at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
      	at hudson.remoting.Channel.call(Channel.java:830)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
      	at sun.reflect.GeneratedMethodAccessor769.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
      	at com.sun.proxy.$Proxy120.execute(Unknown Source)
      	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:804)
      	... 13 more
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: null
      Finished: FAILURE
      

          [JENKINS-44773] User Scoped credentials are not used by the "git" pipeline step

          Mark Waite added a comment -

          If this problem is visible in pipeline, I suspect a similar problem exists outside pipeline as well. Refer to JENKINS-43022 for another bug in git plugin credential scope handling.

          Mark Waite added a comment - If this problem is visible in pipeline, I suspect a similar problem exists outside pipeline as well. Refer to JENKINS-43022 for another bug in git plugin credential scope handling.

          Jesse Glick added a comment -

           i assumed that it was the credentials plugin that was providing the high level abstraction to retrieve credentials

          It does. But it is up to each plugin using credentials to make the correct API calls. Some do, some do not.

          Jesse Glick added a comment -  i assumed that it was the credentials plugin that was providing the high level abstraction to retrieve credentials It does. But it is up to each plugin using credentials to make the correct API calls. Some do, some do not.

          Patrick Wolf added a comment -

          Does the checkout step behave properly? This is the most important question since this is the preferred way of doing checkouts. Can we reproduce same behavior?

          Patrick Wolf added a comment - Does the checkout step behave properly? This is the most important question since this is the preferred way of doing checkouts. Can we reproduce same behavior?

          Jesse Glick added a comment -

          The behavior of checkout is almost certainly the same, since the git step is merely syntactic sugar and winds up calling the same code.

          Jesse Glick added a comment - The behavior of checkout is almost certainly the same, since the git step is merely syntactic sugar and winds up calling the same code.

          We are experiencing a similar issue with Folder scoped credentials.

          Anyone trying to fix this?

          Hugo van den Brand added a comment - We are experiencing a similar issue with Folder scoped credentials. Anyone trying to fix this?

          Mark Waite added a comment -

          I am not trying to fix this at this time hvdbrand.

          Mark Waite added a comment - I am not trying to fix this at this time hvdbrand .

          Matt Sicker added a comment -

          This feature can be replaced by JENKINS-58170 which provides a more standardized approach to granting user-scoped credentials to builds.

          Matt Sicker added a comment - This feature can be replaced by JENKINS-58170 which provides a more standardized approach to granting user-scoped credentials to builds.

          Matt Sicker added a comment -

          There is now a new way to access user-scoped credentials detailed in JENKINS-58170 which uses credentials build parameters. These allow user-scoped credentials to be referenced in any context where you'd normally use the credentials id.

          Matt Sicker added a comment - There is now a new way to access user-scoped credentials detailed in JENKINS-58170 which uses credentials build parameters. These allow user-scoped credentials to be referenced in any context where you'd normally use the credentials id.

          Marek Dlugajczyk added a comment - - edited

          So I have a question. We are using the checkout step in our pipelines to clone, get all the metadata about repo and see the details about repositories in Jenkins GUI. Also checkout step is the recommended to use.

          In our case we have Pipelines that have credentials build parameter of type UsernamePassword (as we clone repos over https) and we require those credentials to be user scoped. Unfortunately when we try to use user scoped credentials with the checkout step it can't clone our private repositories from Gitlab.

          I've tried to use the credentials ID that comes from the build parameter and also tried the build parameter name instead of ID as described in https://issues.jenkins.io/browse/JENKINS-58170 however this does not work as in case of the withCredentials block. To be clear, with the withCredentials block I had no issues to use user scoped credentials.

           

          Consider we have credentials build parameter of type UsernamePassword named in GUI: RepoCredentials - these must be/use user scoped credentials in the build.

          Case 1 - we use the ID of the credentials by passing the value of parameter selected from GUI to checkout step:

          (...)
          script {
              scmVars = checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [
                  [name: params.RepoRef]
              ], doGenerateSubmoduleConfigurations: false, extensions: [
                  [$class: 'WipeWorkspace'],
                  [$class: 'AuthorInChangelog'],
              ], submoduleCfg: [], userRemoteConfigs: [
                  [url: params.RepoUrl, credentialsId: params.RepoCredentials]
              ]]
          }
          (...)

          This does not work properly and we get: 

          [Pipeline] checkout
           Wiping out workspace first.
           Cloning the remote Git repository
           Cloning repository https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git
            > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10
           Fetching upstream changes from https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git
            > git --version # timeout=10
            > git --version # 'git version 2.17.1'
            > git fetch --tags --progress -- https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10
           ERROR: Error cloning remote repo 'origin'
           hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
           stdout: 
           stderr: remote: HTTP Basic: Access denied
           fatal: Authentication failed for 'https://gitlabhost.com/cicd/ansible/playbooks/example-repository.git/'
           
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799)
               at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
               at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
               at hudson.remoting.UserRequest.perform(UserRequest.java:211)
               at hudson.remoting.UserRequest.perform(UserRequest.java:54)
               at hudson.remoting.Request$2.run(Request.java:376)
               at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
               at java.util.concurrent.FutureTask.run(FutureTask.java:266)
               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
               at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
               at java.lang.Thread.run(Thread.java:748)
               Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:49712
                   at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
                   at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
                   at hudson.remoting.Channel.call(Channel.java:1001)
                   at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
                   at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source)
                   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                   at java.lang.reflect.Method.invoke(Method.java:498)
                   at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
                   at com.sun.proxy.$Proxy393.execute(Unknown Source)
                   at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222)
                   at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300)
                   at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125)
                   at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93)
                   at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80)
                   at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
                   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                   ... 1 more

          (I replaced Gitlab URL and repo name for example values)

           

          Case 2 - we use the name of the credentials parameter by passing the name to checkout step as suggested in https://issues.jenkins.io/browse/JENKINS-58170:

          (...)
          script {
              scmVars = checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [
                  [name: params.RepoRef]
              ], doGenerateSubmoduleConfigurations: false, extensions: [
                  [$class: 'WipeWorkspace'],
                  [$class: 'AuthorInChangelog'],
              ], submoduleCfg: [], userRemoteConfigs: [
                  [url: params.RepoUrl, credentialsId: 'RepoCredentials']
              ]]
          }
          (...)
          

          But again that does not work that way and we get:

          [Pipeline] checkout
           Wiping out workspace first.
           Cloning the remote Git repository
           Cloning repository https://githubhost.com/cicd/ansible/playbooks/example-repository.git
            > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10
           Fetching upstream changes from https://githubhost.com/cicd/ansible/playbooks/example-repository.git
            > git --version # timeout=10
            > git --version # 'git version 2.17.1'
            > git fetch --tags --progress -- https://githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10
           ERROR: Error cloning remote repo 'origin'
           hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https://githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
           stdout: 
           stderr: remote: HTTP Basic: Access denied
           fatal: Authentication failed for 'https://githubhost.com/cicd/ansible/playbooks/example-repository.git/'
           
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570)
               at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799)
               at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
               at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
               at hudson.remoting.UserRequest.perform(UserRequest.java:211)
               at hudson.remoting.UserRequest.perform(UserRequest.java:54)
               at hudson.remoting.Request$2.run(Request.java:376)
               at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
               at java.util.concurrent.FutureTask.run(FutureTask.java:266)
               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
               at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
               at java.lang.Thread.run(Thread.java:748)
               Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:42646
                   at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
                   at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
                   at hudson.remoting.Channel.call(Channel.java:1001)
                   at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
                   at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source)
                   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                   at java.lang.reflect.Method.invoke(Method.java:498)
                   at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
                   at com.sun.proxy.$Proxy393.execute(Unknown Source)
                   at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222)
                   at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300)
                   at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125)
                   at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93)
                   at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80)
                   at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
                   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                   ... 1 more

          (btw. I double checked the credentials configured in Jenkins to make sure there are no typos etc. as we got: "returned status code 128", "HTTP Basic: Access denied")

           

          So in general my question is how we can use user scoped credentials with the checkout step the correct way? jvz you wrote that it is possible to access user scoped credentials in: "any context where you'd normally use the credentials id." Could you explain what I'm missing here?

           

          I also want to add that I've found a workaround for this but it's not pretty IMHO. And I'm really curious how it should be done correctly. I'll post the workaround in separate comment as it might be useful.

          Marek Dlugajczyk added a comment - - edited So I have a question. We are using the checkout step in our pipelines to clone, get all the metadata about repo and see the details about repositories in Jenkins GUI. Also checkout step is the recommended to use. In our case we have Pipelines that have credentials build parameter of type UsernamePassword (as we clone repos over https) and we require those credentials to be user scoped . Unfortunately when we try to use user scoped credentials with the checkout step it can't clone our private repositories from Gitlab. I've tried to use the credentials ID that comes from the build parameter and also tried the build parameter name instead of ID as described in https://issues.jenkins.io/browse/JENKINS-58170  however this does not work as in case of the withCredentials block. To be clear, with the withCredentials block I had no issues to use user scoped credentials.   Consider we have credentials build parameter of type UsernamePassword named in GUI: RepoCredentials - these must be/use user scoped credentials in the build. Case 1 - we use the ID of the credentials by passing the value of parameter selected from GUI to checkout step: (...) script { scmVars = checkout changelog: true , poll: true , scm: [$class: 'GitSCM' , branches: [ [name: params.RepoRef] ], doGenerateSubmoduleConfigurations: false , extensions: [ [$class: 'WipeWorkspace' ], [$class: 'AuthorInChangelog' ], ], submoduleCfg: [], userRemoteConfigs: [ [url: params.RepoUrl, credentialsId: params.RepoCredentials] ]] } (...) This does not work properly and we get:  [Pipeline] checkout Wiping out workspace first. Cloning the remote Git repository Cloning repository https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10 Fetching upstream changes from https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git > git --version # timeout=10 > git --version # 'git version 2.17.1' > git fetch --tags --progress -- https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10 ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: remote: HTTP Basic: Access denied fatal: Authentication failed for 'https: //gitlabhost.com/cicd/ansible/playbooks/example-repository.git/' at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:376) at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119) at java.lang. Thread .run( Thread .java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:49712 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy393.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more (I replaced Gitlab URL and repo name for example values)   Case 2 - we use the name of the credentials parameter by passing the name to checkout step as suggested in https://issues.jenkins.io/browse/JENKINS-58170: (...) script { scmVars = checkout changelog: true , poll: true , scm: [$class: 'GitSCM' , branches: [ [name: params.RepoRef] ], doGenerateSubmoduleConfigurations: false , extensions: [ [$class: 'WipeWorkspace' ], [$class: 'AuthorInChangelog' ], ], submoduleCfg: [], userRemoteConfigs: [ [url: params.RepoUrl, credentialsId: 'RepoCredentials' ] ]] } (...) But again that does not work that way and we get: [Pipeline] checkout Wiping out workspace first. Cloning the remote Git repository Cloning repository https: //githubhost.com/cicd/ansible/playbooks/example-repository.git > git init /home/tfchefrunner/agent/workspace/DEVOPS_Generic_CM # timeout=10 Fetching upstream changes from https: //githubhost.com/cicd/ansible/playbooks/example-repository.git > git --version # timeout=10 > git --version # 'git version 2.17.1' > git fetch --tags --progress -- https: //githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/* # timeout=10 ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "git fetch --tags --progress -- https: //githubhost.com/cicd/ansible/playbooks/example-repository.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: remote: HTTP Basic: Access denied fatal: Authentication failed for 'https: //githubhost.com/cicd/ansible/playbooks/example-repository.git/' at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2608) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2048) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:570) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:799) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:376) at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119) at java.lang. Thread .run( Thread .java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.0.0/10.244.0.0:42646 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1001) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor2254.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy393.execute(Unknown Source) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1300) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:93) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:80) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more (btw. I double checked the credentials configured in Jenkins to make sure there are no typos etc. as we got: "returned status code 128", "HTTP Basic: Access denied" )   So in general my question is how we can use user scoped credentials with the checkout step the correct way? jvz  you wrote that it is possible to access user scoped credentials in: "any context where you'd normally use the credentials id." Could you explain what I'm missing here?   I also want to add that I've found a workaround for this but it's not pretty IMHO. And I'm really curious how it should be done correctly. I'll post the workaround in separate comment as it might be useful.

          Marek Dlugajczyk added a comment - - edited

          As mentioned above I'm posting my "ugly" workaround for using user scoped credentials with checkout step:

           

          Consider we have credentials build parameter of type UsernamePassword named in GUI: RepoCredentials - these must be/use user scoped credentials in the build.

          (...)
          stage('Clone') {
              steps {
                  retry(3) {
                      script {
                          /**
                           * All steps below are temporary workaround for user scoped credentials not being resolved by the
                           * checkout/git steps, see https://issues.jenkins.io/browse/JENKINS-44773
                           * Once this will work properly in Jenkins we can just do simple checkout step as normal, until
                           * that time we need to use withCredentials as this can resolve user scoped credentials without a problem
                           * Tried using GIT_ASKPASS method here as well but that didn't work at all in checkout step
                           * We still want to use the checkout step for repo details in Jenkins GUI that this step sets
                           */
                          withCredentials([
                              usernamePassword(credentialsId: 'RepoCredentials', usernameVariable: 'GIT_USR', passwordVariable: 'GIT_PSW')
                          ]) {                    
          
                              def tmpGitPassFilePath = "${pwd(tmp: true)}/.git-credentials"
                              URL projectURL = new URL(params.RepoUrl)                    
                              
                              /**
                               * Using writeFile as echoing this in 'sh' step would print contents in BlueOcean
                               * We need to encode Git password if it has special characters, encoded spaces '+' sign
                               * must be converted to '%20'
                               */
                              def encodedPsw = URLEncoder.encode(GIT_PSW, 'UTF-8').replace('+', '%20')
                              writeFile file: tmpGitPassFilePath, text: """\
                                  ${projectURL.getProtocol()}://${GIT_USR}:${encodedPsw}@${projectURL.getHost()}
                                  """.stripIndent()                    
          
                              sh label: 'Configure local Git before checkout', script: """
                                  #!/bin/bash +x
                                  git config --global credential.helper 'store --file=${tmpGitPassFilePath}'
                                  """.stripIndent().trim()                    
                              
                              scmVars = checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [
                                  [name: params.RepoRef]
                              ], doGenerateSubmoduleConfigurations: false, extensions: [
                                  [$class: 'WipeWorkspace'],
                                  [$class: 'AuthorInChangelog'],
                              ], submoduleCfg: [], userRemoteConfigs: [
                                  /**
                                   * No credentialsID provided to use our local git temp. config
                                   * see https://github.com/jenkinsci/git-plugin/blob/de97701cb452c20f967072a1cab99ad2efb96425/src/main/java/jenkins/plugins/git/GitSCMBuilder.java#L116
                                   */
                                  [url: params.RepoUrl]
                              ]]                    
          
                              sh label: 'Clear temporary Git config after checkout', script: """
                                  #!/bin/bash +x
                                  git config --global --unset credential.helper
                                  rm -f ${tmpGitPassFilePath}
                                  """.stripIndent().trim()
                          }
                      }
                  }
              }
          }

          Marek Dlugajczyk added a comment - - edited As mentioned above I'm posting my "ugly" workaround for using user scoped credentials with checkout step:   Consider we have credentials build parameter of type UsernamePassword named in GUI:  RepoCredentials  - these must be/use user scoped credentials in the build. (...) stage( 'Clone' ) { steps { retry(3) { script { /** * All steps below are temporary workaround for user scoped credentials not being resolved by the * checkout/git steps, see https: //issues.jenkins.io/browse/JENKINS-44773 * Once this will work properly in Jenkins we can just do simple checkout step as normal, until * that time we need to use withCredentials as this can resolve user scoped credentials without a problem * Tried using GIT_ASKPASS method here as well but that didn't work at all in checkout step * We still want to use the checkout step for repo details in Jenkins GUI that this step sets */ withCredentials([ usernamePassword(credentialsId: 'RepoCredentials' , usernameVariable: 'GIT_USR' , passwordVariable: 'GIT_PSW' ) ]) { def tmpGitPassFilePath = "${pwd(tmp: true )}/.git-credentials" URL projectURL = new URL(params.RepoUrl) /** * Using writeFile as echoing this in 'sh' step would print contents in BlueOcean * We need to encode Git password if it has special characters, encoded spaces '+' sign * must be converted to '%20' */ def encodedPsw = URLEncoder.encode(GIT_PSW, 'UTF-8' ).replace( '+' , '%20' ) writeFile file: tmpGitPassFilePath, text: """\ ${projectURL.getProtocol()}: //${GIT_USR}:${encodedPsw}@${projectURL.getHost()} """.stripIndent() sh label: 'Configure local Git before checkout' , script: """ #!/bin/bash +x git config --global credential.helper 'store --file=${tmpGitPassFilePath}' """.stripIndent().trim() scmVars = checkout changelog: true , poll: true , scm: [$class: 'GitSCM' , branches: [ [name: params.RepoRef] ], doGenerateSubmoduleConfigurations: false , extensions: [ [$class: 'WipeWorkspace' ], [$class: 'AuthorInChangelog' ], ], submoduleCfg: [], userRemoteConfigs: [ /** * No credentialsID provided to use our local git temp. config * see https: //github.com/jenkinsci/git-plugin/blob/de97701cb452c20f967072a1cab99ad2efb96425/src/main/java/jenkins/plugins/git/GitSCMBuilder.java#L116 */ [url: params.RepoUrl] ]] sh label: 'Clear temporary Git config after checkout' , script: """ #!/bin/bash +x git config --global --unset credential.helper rm -f ${tmpGitPassFilePath} """.stripIndent().trim() } } } } }

            Unassigned Unassigned
            cleclerc Cyrille Le Clerc
            Votes:
            6 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: