Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28335

Pipeline step to run Git commands with credentials & tool

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • git-plugin
    • 4.8.0 - released username / password credential binding

      It would be nice to be able to use the GitPublisher inside a workflow as it is possible in other jobs.

      This requires the plugin to be upgrade to Jenkins core 1.580.1+, to implement the jenkins.tasks.SimpleBuildStep in GitPublisher

          [JENKINS-28335] Pipeline step to run Git commands with credentials & tool

          medianick I've now made it work by converting the job to pipeline and using the withCredentials directive, thereby injecting the credentials into the url myself. I never found a way to access the credentials to do the same with my previous freestyle project.

          But what I don't understand is why we must do this credential fetching and url massaging ourselves, i.e. why for instance djviking's example using the GitClient class directly doesn't (seem to) work.

          Jesper Matthiesen added a comment - medianick I've now made it work by converting the job to pipeline and using the  withCredentials directive, thereby injecting the credentials into the url myself. I never found a way to access the credentials to do the same with my previous freestyle project. But what I don't understand is why we must do this credential fetching and url massaging ourselves, i.e. why for instance djviking 's example using the GitClient class directly doesn't (seem to) work.

          Michael Beaumont added a comment - - edited

          I think I've come up with a good workaround. We can set the following at the beginning of our build (declarative pipeline in a Github organization):

          sh 'git config --local credential.helper "!p() { echo username=\\$GIT_USERNAME; echo password=\\$GIT_PASSWORD; }; p"'
          

          See git credential helpers

           

          Then, when we want to use the credentials we can use a block like the following: 

          sh 'git tag -m "" ${VERSION_NUMBER}'
          withCredentials([
            usernamePassword(credentialsId: 'github', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD')
          ]) {
            sh 'git push origin ${VERSION_NUMBER}'
          }
          

          This way we don't have to even repeat the URL for the origin remote, which is already set. 

          Michael Beaumont added a comment - - edited I think I've come up with a good workaround. We can set the following at the beginning of our build (declarative pipeline in a Github organization): sh 'git config --local credential.helper "!p() { echo username=\\$GIT_USERNAME; echo password=\\$GIT_PASSWORD; }; p" ' See git credential helpers   Then, when we want to use the credentials we can use a block like the following:  sh 'git tag -m "" ${VERSION_NUMBER}' withCredentials([ usernamePassword(credentialsId: 'github' , usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' ) ]) { sh 'git push origin ${VERSION_NUMBER}' } This way we don't have to even repeat the URL for the origin remote, which is already set. 

          Kari Niemi added a comment -

          Official coudbees provided instructions for the workaround: https://support.cloudbees.com/hc/en-us/articles/360027646491-Pipeline-Equivalent-to-Git-Publisher

          ...but they have bugs in the examples: withCredentials is used to publish variable GIT_USERNAME  ... and then that variable is referenced with GIT_AUTH_USR in shell step. The same mistake for password variables.

          Kari Niemi added a comment - Official coudbees provided instructions for the workaround: https://support.cloudbees.com/hc/en-us/articles/360027646491-Pipeline-Equivalent-to-Git-Publisher ...but they have bugs in the examples: withCredentials is used to publish variable GIT_USERNAME  ... and then that variable is referenced with GIT_AUTH_USR in shell step. The same mistake for password variables.

          Mark Waite added a comment -

          Git plugin 4.8.0 released July 21, 2021 with the git credentials binding for username / password credentials. Special thanks to Harshit Chopra, the Google Summer of Code 2021 student that implemented it.

          Work continues on the git credentials binding for ssh private key credentials. Planned to be included in a future release of the git plugin.

          Mark Waite added a comment - Git plugin 4.8.0 released July 21, 2021 with the git credentials binding for username / password credentials. Special thanks to Harshit Chopra, the Google Summer of Code 2021 student that implemented it. Work continues on the git credentials binding for ssh private key credentials. Planned to be included in a future release of the git plugin.

          Dee Kryvenko added a comment -

          New gitUsernamePassword binding only accounts for a single set of credentials.

          Often there are different set of credentials required for different git locations.

          Examples would be Go Modules, Terraform Modules and other that are using Git for dependency management. If the current repository we are building have dependencies on other repositories - we need read-write git credentials to the current repository (so we could publish tags, for example) while we need read-only set of credentials to everything else for fetching dependencies.

          If this binding currently implemented as AskPass helper - an easy fix would be to accept additional parameter in the binding that'd tell the location the binding is for.

          Dee Kryvenko added a comment - New gitUsernamePassword binding only accounts for a single set of credentials. Often there are different set of credentials required for different git locations. Examples would be Go Modules, Terraform Modules and other that are using Git for dependency management. If the current repository we are building have dependencies on other repositories - we need read-write git credentials to the current repository (so we could publish tags, for example) while we need read-only set of credentials to everything else for fetching dependencies. If this binding currently implemented as AskPass helper - an easy fix would be to accept additional parameter in the binding that'd tell the location the binding is for.

          Mark Waite added a comment -

          llibicpep couldn't you achieve the same result by having a single credential that is used for read access to all repositories and an additional credential that allows read/write access to the one repository that needs to be written? Read operations (like fetch and clone) would be performed inside a withCredentials block using the read credentials, then a separate withCredentials block would be used to perform the write operations.

          Mark Waite added a comment - llibicpep couldn't you achieve the same result by having a single credential that is used for read access to all repositories and an additional credential that allows read/write access to the one repository that needs to be written? Read operations (like fetch and clone) would be performed inside a withCredentials block using the read credentials, then a separate withCredentials block would be used to perform the write operations.

          Dee Kryvenko added a comment -

          markewaite - you are assuming entire pipeline is a Jenkinsfile. That is far from being even in majority of the cases. In reality - Jenkinsfile delegates to the tools such as Make/Maven/Gradle/NPM/Terraform/etc. User-supplied untrusted code should be able to safely access only what's it allowed to - write to the current repository and read other repositories as dependencies. From the Jenkinsfile level - we don't have exposure into what is going to happen in the user-supplied code. To enable this workflow - we need to pass in to the tool several sets of credentials. And it might be even more complicated as there could be several locations such as GitHub Orgs in play with different sets of credentials each - or even entirely different providers (GitHub+GitLab for example).

          Dee Kryvenko added a comment - markewaite - you are assuming entire pipeline is a Jenkinsfile. That is far from being even in majority of the cases. In reality - Jenkinsfile delegates to the tools such as Make/Maven/Gradle/NPM/Terraform/etc. User-supplied untrusted code should be able to safely access only what's it allowed to - write to the current repository and read other repositories as dependencies. From the Jenkinsfile level - we don't have exposure into what is going to happen in the user-supplied code. To enable this workflow - we need to pass in to the tool several sets of credentials. And it might be even more complicated as there could be several locations such as GitHub Orgs in play with different sets of credentials each - or even entirely different providers (GitHub+GitLab for example).

          Mark Waite added a comment -

          llibicpep since the withCredentials step provides the credential to all git operations inside the sh, bat, or powershell step, I would expect it to be well behaved whether the program that called git was make, maven, gradle, bazel, or any other tool. I think your scenario of mapping different credentials to individual repositories sounds like a more challenging use case than I'm ready to solve using the withCredentials technique.

          Mark Waite added a comment - llibicpep since the withCredentials step provides the credential to all git operations inside the sh , bat , or powershell step, I would expect it to be well behaved whether the program that called git was make, maven, gradle, bazel, or any other tool. I think your scenario of mapping different credentials to individual repositories sounds like a more challenging use case than I'm ready to solve using the withCredentials technique.

          Dee Kryvenko added a comment - - edited

          markewaite the "program" might want to fetch dependencies AND push a release tag, as an example, all in the same time. You don't get to know that from your Jenkinsfile. In the Jenkinsfile it is a single step.

          It's all quite simple actually - the new bindings could accept multiple pairs url:credential-id which would render into a config file like this:

          [credential "https://github.com"]
          	useHttpPath = true
          	helper = ...
          [credential "https://gitlab.com"]
          	useHttpPath = true
          	helper = ...
          

          And the helper then would look into both host AND path in the input (which the latter would only be available when useHttpPath = true) to match with a UsernamePasswordCredentials instance.

          Dee Kryvenko added a comment - - edited markewaite the "program" might want to fetch dependencies AND push a release tag, as an example, all in the same time. You don't get to know that from your Jenkinsfile. In the Jenkinsfile it is a single step. It's all quite simple actually - the new bindings could accept multiple pairs url:credential-id which would render into a config file like this: [credential "https: //github.com" ] useHttpPath = true helper = ... [credential "https: //gitlab.com" ] useHttpPath = true helper = ... And the helper then would look into both host AND path in the input (which the latter would only be available when useHttpPath = true ) to match with a UsernamePasswordCredentials instance.

          David Url added a comment -

          Would this feature work as well in stages that run in containers?

          David Url added a comment - Would this feature work as well in stages that run in containers?

            Unassigned Unassigned
            alecharp Adrien Lecharpentier
            Votes:
            160 Vote for this issue
            Watchers:
            165 Start watching this issue

              Created:
              Updated: