Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28335

Pipeline step to run Git commands with credentials & tool

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • git-plugin
    • 4.8.0 - released username / password credential binding

      It would be nice to be able to use the GitPublisher inside a workflow as it is possible in other jobs.

      This requires the plugin to be upgrade to Jenkins core 1.580.1+, to implement the jenkins.tasks.SimpleBuildStep in GitPublisher

          [JENKINS-28335] Pipeline step to run Git commands with credentials & tool

          Code changed in jenkins
          User: Jesse Glick
          Path:
          COMPATIBILITY.md
          http://jenkins-ci.org/commit/workflow-plugin/26beaf00bff3fc235cb73d53c77bf64b632fdc75
          Log:
          JENKINS-28335 Noting.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: COMPATIBILITY.md http://jenkins-ci.org/commit/workflow-plugin/26beaf00bff3fc235cb73d53c77bf64b632fdc75 Log: JENKINS-28335 Noting.

          Sverre Moe added a comment - - edited

          Meanwhile it is possible to use Groovy Post-build within Pipeline (formerly Workflow).

          Using this script one could publish git tags:

          def build = manager.build
          def workspace = build.getWorkspace()
          def listener = manager.listener
          def environment = build.getEnvironment(listener)
          
          final def project = build.getProject()
          final def gitScm = project.getScm()
          final GitClient gitClient = gitScm.createClient(listener, environment, build, workspace);
          
          final def gitTagName = "TAG_NAME"
          final def comment = "COMMENT"
          final def remoteURI = new URIish("origin")
          
          gitClient.tag(gitTagName, comment)
          gitClient.push().tags(true).to(remoteURI).execute()
          

          Sverre Moe added a comment - - edited Meanwhile it is possible to use Groovy Post-build within Pipeline (formerly Workflow). Using this script one could publish git tags: def build = manager.build def workspace = build.getWorkspace() def listener = manager.listener def environment = build.getEnvironment(listener) final def project = build.getProject() final def gitScm = project.getScm() final GitClient gitClient = gitScm.createClient(listener, environment, build, workspace); final def gitTagName = "TAG_NAME" final def comment = "COMMENT" final def remoteURI = new URIish( "origin" ) gitClient.tag(gitTagName, comment) gitClient.push().tags( true ).to(remoteURI).execute()

          Alexander Siniouguine added a comment - - edited

          djviking How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs (the actual jenkinsfile is here: https://github.com/VirtoCommerce/vc-module-core/blob/eeee111fb33e12f93c9bf64988a193627dfb533a/Jenkinsfile)

          Ideally I'd like to commit the file changed during the build (version file).

          Alexander Siniouguine added a comment - - edited djviking How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs (the actual jenkinsfile is here: https://github.com/VirtoCommerce/vc-module-core/blob/eeee111fb33e12f93c9bf64988a193627dfb533a/Jenkinsfile ) Ideally I'd like to commit the file changed during the build (version file).

          Art V added a comment -

          I also had issues with this.. What I did is use the sshagent plugin and ran the git tag commands inside this. It may not be as flexible/robust as above, but an easier straight forward workaround until we have a better solution to get the GitSCM client. I have not done much testing but hope this helps you in the meantime.

          def call(String tag, String comment, String credentialsId, String repoName = 'origin') {
              sshagent([credentialsId]) {
                  sh("git tag -a -f -m '${comment}' ${tag}")
                  sh("git -c core.askpass=true push ${repoName} ${tag}")
              }
          }
          

          Art V added a comment - I also had issues with this.. What I did is use the sshagent plugin and ran the git tag commands inside this. It may not be as flexible/robust as above, but an easier straight forward workaround until we have a better solution to get the GitSCM client. I have not done much testing but hope this helps you in the meantime. def call( String tag, String comment, String credentialsId, String repoName = 'origin' ) { sshagent([credentialsId]) { sh( "git tag -a -f -m '${comment}' ${tag}" ) sh( "git -c core.askpass= true push ${repoName} ${tag}" ) } }

          Sverre Moe added a comment -

          How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs

          The manager object should be available if you have installed the Groovy Postbuild Plugin.

          Sverre Moe added a comment - How do you run postbuild from within jenkinsfile itself, the "manager" object is not available when it runs The manager object should be available if you have installed the Groovy Postbuild Plugin.

          mocsharp added a comment - - edited

          sverre_boschman I tried the code you've provided but am getting the following error:

          Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace
          
          org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace
          	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:113)
          	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
          	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
          	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:15)
          	at WorkflowScript.stageTagBuild(WorkflowScript:234)
          	at WorkflowScript.run(WorkflowScript:52)
          	at ___cps.transform___(Native Method)
          	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:55)
          	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
          	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:74)
          	at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
          	at java.lang.reflect.Method.invoke(Unknown Source)
          	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
          	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
          	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
          	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
          	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
          	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:32)
          	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:29)
          	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
          	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:29)
          	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:297)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:78)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:206)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:204)
          	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
          	at java.util.concurrent.FutureTask.run(Unknown Source)
          	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
          	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
          	at java.util.concurrent.FutureTask.run(Unknown Source)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          	at java.lang.Thread.run(Unknown Source)
          

          Any idea? Thanks!

          Edit: nvm. I used pwd() instead.

          Edit: getting the following as well:

          Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getProject
          

          mocsharp added a comment - - edited sverre_boschman I tried the code you've provided but am getting the following error: Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getWorkspace at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:15) at WorkflowScript.stageTagBuild(WorkflowScript:234) at WorkflowScript.run(WorkflowScript:52) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:55) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:74) at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:58) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:32) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:29) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:29) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:297) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:78) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:206) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:204) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47) at java.util.concurrent.FutureTask.run(Unknown Source) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Any idea? Thanks! Edit: nvm. I used pwd() instead. Edit: getting the following as well: Build Failed: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: unclassified method org.jenkinsci.plugins.workflow.job.WorkflowRun getProject

          The workaround published in https://github.com/jenkinsci/pipeline-examples/blob/master/pipeline-examples/push-git-repo/pushGitRepo.Groovy doesn't work with special characters in the password (e.g. @). You can get around that by URL encoding the password:

          withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'MyID', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD']]) {
              String encoded_password = java.net.URLEncoder.encode(env.GIT_PASSWORD, "UTF-8")
              sh("git tag -a some_tag -m 'Jenkins'")
              sh("git push https://${env.GIT_USERNAME}:${encoded_password}@<REPO> --tags")
          }
          

          but this defeats the credential binding attempts to obscure the password in the console output. Could do with a proper fix.

          Russell Gallop added a comment - The workaround published in https://github.com/jenkinsci/pipeline-examples/blob/master/pipeline-examples/push-git-repo/pushGitRepo.Groovy doesn't work with special characters in the password (e.g. @). You can get around that by URL encoding the password: withCredentials([[$class: 'UsernamePasswordMultiBinding' , credentialsId: 'MyID' , usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' ]]) { String encoded_password = java.net.URLEncoder.encode(env.GIT_PASSWORD, "UTF-8" ) sh( "git tag -a some_tag -m 'Jenkins' " ) sh( "git push https: //${env.GIT_USERNAME}:${encoded_password}@<REPO> --tags" ) } but this defeats the credential binding attempts to obscure the password in the console output. Could do with a proper fix.

          +1

          Mayur Barge added a comment -

          Hi jglick
          Greetings !
          Are there any plans to get GitPublisher working soon ?
          OR to fix the special characters issue mentioned by rg

          Thanks rg for the workaround. But as you mentioned it shows password in the console output. Any other suggestions ?

          Mayur Barge added a comment - Hi jglick Greetings ! Are there any plans to get GitPublisher working soon ? OR to fix the special characters issue mentioned by rg Thanks rg for the workaround. But as you mentioned it shows password in the console output. Any other suggestions ?

          Hello,
          need the pipeline functioning ASAP
          could you please provide such feature?
          thanks

          Viachaslau Kabak added a comment - Hello, need the pipeline functioning ASAP could you please provide such feature? thanks

          Daniel Sobral added a comment -

          Workaround:

              withCredentials([[$class  : 'FileBinding', credentialsId: 'your-credential-id',
                                variable: 'CREDENTIALS']]) {
                  sh 'git config --local credential.username RepoUserName'
                  sh "git config --local credential.helper 'store --file=${env.CREDENTIALS}'"
          
                  sh "git do-your-stuff"
          
                  sh 'git config --local --remove-section credential'
              }
          

          See git man page for the git store credential for information on the file format.

          Daniel Sobral added a comment - Workaround: withCredentials([[$class : 'FileBinding' , credentialsId: 'your-credential-id' , variable: 'CREDENTIALS' ]]) { sh 'git config --local credential.username RepoUserName' sh "git config --local credential.helper 'store --file=${env.CREDENTIALS}' " sh "git do -your-stuff" sh 'git config --local --remove-section credential' } See git man page for the git store credential for information on the file format.

          Daniel Sobral added a comment -

          Mind you, it doesn't work for submodules. Submodules have a fundamental flaw: they are repos on their own, so the credentials for the module that include them does not apply, but you can't set local credentials for them before they are fetched. Catch-22.

          For submodules, either use relative paths, SSH keys, or run a sed on the URL of .gitmodules.

          Daniel Sobral added a comment - Mind you, it doesn't work for submodules. Submodules have a fundamental flaw: they are repos on their own, so the credentials for the module that include them does not apply, but you can't set local credentials for them before they are fetched. Catch-22. For submodules, either use relative paths, SSH keys, or run a sed on the URL of .gitmodules.

          Dries De Smet added a comment - - edited

          dcsobral Do you by any chance know why the following script: (which I found as an alternative to the one above), gives the following error?

          fatal: repository 'https://****:****@github.com/ourorg/therepo.git/' not found
          
          withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '48bfc941-310e-4a0b-bf87-15bc4c4e06cd',
                                      usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
                  sh "git checkout develop"
                  sh "git checkout master"
                  sh "git merge develop"
                  sh "git tag " + versionLabel
                  sh "git push 'https://${env.USERNAME}:${env.PASSWORD}@github.com/ourorg/therepo.git' --tags"
              }
          

          Locally, the command works fine with the ' , I triple checked spelling & tried different quotes, but in the pipeline it keeps saying that it can't find the repo.

          Dries De Smet added a comment - - edited dcsobral Do you by any chance know why the following script: (which I found as an alternative to the one above), gives the following error? fatal: repository 'https: //****:****@github.com/ourorg/therepo.git/' not found withCredentials([[$class: 'UsernamePasswordMultiBinding' , credentialsId: '48bfc941-310e-4a0b-bf87-15bc4c4e06cd' , usernameVariable: 'USERNAME' , passwordVariable: 'PASSWORD' ]]) { sh "git checkout develop" sh "git checkout master" sh "git merge develop" sh "git tag " + versionLabel sh "git push 'https: //${env.USERNAME}:${env.PASSWORD}@github.com/ourorg/therepo.git' --tags" } Locally, the command works fine with the ' , I triple checked spelling & tried different quotes, but in the pipeline it keeps saying that it can't find the repo.

          Ruslan Bondarau added a comment - - edited
          withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '9e5r57b0-5649-3d65-rt41-e900d8c76774', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME']])
          {
              sh "git tag -a some_tag${env.BUILD_NUMBER} -m 'Jenkins'"
              sh ("git -c core.askpass=true push https://${env.GIT_USERNAME}:${env.GIT_PASSWORD}@github.com/some-tag/some-tag.git some_tag${env.BUILD_NUMBER}")
          }
          

          Ruslan Bondarau added a comment - - edited withCredentials([[$class: 'UsernamePasswordMultiBinding' , credentialsId: '9e5r57b0-5649-3d65-rt41-e900d8c76774' , passwordVariable: 'GIT_PASSWORD' , usernameVariable: 'GIT_USERNAME' ]]) { sh "git tag -a some_tag${env.BUILD_NUMBER} -m 'Jenkins' " sh ( "git -c core.askpass= true push https: //${env.GIT_USERNAME}:${env.GIT_PASSWORD}@github.com/some-tag/some-tag.git some_tag${env.BUILD_NUMBER}" ) }

          There's a workaround using ssh-agent-plugin:

          sshagent(['git-credentials-id']) {
                sh "git push origin master"
          }
          

          Leonardo Zanivan added a comment - There's a workaround using ssh-agent-plugin: sshagent([ 'git-credentials-id' ]) { sh "git push origin master" }

          Andrey Makeev added a comment - - edited

          I want to share my Jenkins Pipeline setup and my solution to publish changes/tags to git repo via SSH (While this task is under development). Please check it out for more info, any improvement ideas are welcome
          In short you just add file git_push_ssh.groovy to your project and call method pushSSH() from Jenkinsfile like this:

           env.BRANCH_NAME = "mycoolbranch"// BRANCH_NAME is predefined in multibranch pipeline job
           env.J_GIT_CONFIG = "true"
           env.J_USERNAME = "Jenkins CI"
           env.J_EMAIL = "jenkins-ci@example.com"
           env.J_CREDS_IDS = '02aa92ec-593e-4a90-ac85-3f43a06cfae3' // Use credentials id from Jenkins
           def gitLib = load "git_push_ssh.groovy"
           ...
           gitLib.pushSSH(commitMsg: "Jenkins build #${env.BUILD_NUMBER}", tagName: "build-${env.BUILD_NUMBER}", files: "changelog.txt someotherfile.txt");
          

          Andrey Makeev added a comment - - edited I want to share my Jenkins Pipeline setup and my solution to publish changes/tags to git repo via SSH (While this task is under development). Please check it out for more info, any improvement ideas are welcome In short you just add file git_push_ssh.groovy to your project and call method pushSSH() from Jenkinsfile like this: env.BRANCH_NAME = "mycoolbranch" // BRANCH_NAME is predefined in multibranch pipeline job env.J_GIT_CONFIG = " true " env.J_USERNAME = "Jenkins CI" env.J_EMAIL = "jenkins-ci@example.com" env.J_CREDS_IDS = '02aa92ec-593e-4a90-ac85-3f43a06cfae3' // Use credentials id from Jenkins def gitLib = load "git_push_ssh.groovy" ... gitLib.pushSSH(commitMsg: "Jenkins build #${env.BUILD_NUMBER}" , tagName: "build-${env.BUILD_NUMBER}" , files: "changelog.txt someotherfile.txt" );

          cowwoc added a comment -

          Beware: ssh-agent will not currently work under windows: JENKINS-28279 ("git plugin", such as the "checkout" task will fail to authenticate)

          As far as I can tell, the only option under Windows is to pass the username/password to an HTTPS url: http://stackoverflow.com/a/33630506/14731

          cowwoc added a comment - Beware: ssh-agent will not currently work under windows: JENKINS-28279 ("git plugin", such as the "checkout" task will fail to authenticate) As far as I can tell, the only option under Windows is to pass the username/password to an HTTPS url: http://stackoverflow.com/a/33630506/14731

          jim jaeger added a comment -

          Any Updates for the usage of the GitPublisher in JenkinsFiles?

          jim jaeger added a comment - Any Updates for the usage of the GitPublisher in JenkinsFiles?

          James Hogarth added a comment -

          I like the files idea but we needed a user/pass version.

          My contribution and modification to this is:

          1. Our ansible deployment configures:
            - name: configure git username
              become_user: jenkins
              git_config:
                scope: global
                name: user.name
                value: "{{ lookup('env','USER') }}"
            
            - name: configure git email
              become_user: jenkins
              git_config:
                scope: global
                name: user.email
                value: "{{ lookup('env','USER') }}@{{ ansible_hostname }}"
            
            - name: enable git cache credential storage
              become_user: jenkins
              git_config:
                scope: global
                name: credential.helper
                value: cache
            
          2. The jenkins pipeline contains:
            stage("tag the commit with datetime") {
              withCredentials([usernamePassword(credentialsId: 'my_cred_id' usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD')]) {
            
              // use date for tag
              def tag = new Date().format("yyyyMMddHHmm")
            
              // configure the git credentials, these are cached in RAM for several minutes to use
              // this is required until https://issues.jenkins-ci.org/browse/JENKINS-28335 is resolved upstream
              sh "echo 'protocol=https\nhost=<git-host-goes-here>\nusername=${GIT_USERNAME}\npassword=${GIT_PASSWORD}\n\n' | git credential approve "
            
              sh "git tag -a ${tag} -m '${USER} tagging'"
              sh "git push --tags"
              }
            }
            

          This example uses new Date() which needs whitelisting right now but could easily be BUILD_ID if you were sure that you'd never need to rebuild, and lose an incrementing BUILD_ID, or whatever other metric/value was preferred.

          James Hogarth added a comment - I like the files idea but we needed a user/pass version. My contribution and modification to this is: Our ansible deployment configures: - name: configure git username become_user: jenkins git_config: scope: global name: user.name value: "{{ lookup( 'env' , 'USER' ) }}" - name: configure git email become_user: jenkins git_config: scope: global name: user.email value: "{{ lookup( 'env' , 'USER' ) }}@{{ ansible_hostname }}" - name: enable git cache credential storage become_user: jenkins git_config: scope: global name: credential.helper value: cache The jenkins pipeline contains: stage( "tag the commit with datetime" ) { withCredentials([usernamePassword(credentialsId: 'my_cred_id' usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' )]) { // use date for tag def tag = new Date().format( "yyyyMMddHHmm" ) // configure the git credentials, these are cached in RAM for several minutes to use // this is required until https://issues.jenkins-ci.org/browse/JENKINS-28335 is resolved upstream sh "echo 'protocol=https\nhost=<git-host-goes-here>\nusername=${GIT_USERNAME}\npassword=${GIT_PASSWORD}\n\n' | git credential approve " sh "git tag -a ${tag} -m '${USER} tagging' " sh "git push --tags" } } This example uses new Date() which needs whitelisting right now but could easily be BUILD_ID if you were sure that you'd never need to rebuild, and lose an incrementing BUILD_ID, or whatever other metric/value was preferred.

          mocsharp added a comment -

          Thank you hogarthj

           

          For Windows, do

          bat "cmd /c echo protocol=https & echo.host=<git-host-goes-here> & echo.username=${GIT_USERNAME} & echo.password=${GIT_PASSWORD} | git credential approve "

          Note the dot after echo.

          mocsharp added a comment - Thank you hogarthj   For Windows, do bat "cmd /c echo protocol=https & echo.host=<git-host-goes-here> & echo.username=${GIT_USERNAME} & echo.password=${GIT_PASSWORD} | git credential approve " Note the dot after echo.

          Jesse Glick added a comment -

          I would not expect the stated RFE to be implemented as such. Rather, there can be a withGit block-scoped step which takes an installation name and/or credentials ID, defines suitable environment variables, and passes off control to nested steps like sh.

          Jesse Glick added a comment - I would not expect the stated RFE to be implemented as such. Rather, there can be a withGit block-scoped step which takes an installation name and/or credentials ID, defines suitable environment variables, and passes off control to nested steps like sh .

          Nick Jones added a comment - - edited

          Indeed, jglick, what you described is essentially what I ended up doing (in declarative pipeline):

          environment {
            GITUSER = credentials('some_credential_id')
          }
          steps {
            sh 'git tag -a some.tag -m "some message"'
            sh 'git push https://${GITUSER_USR}:${GITUSER_PSW}@some.git.url some.tag'
          }

          I've omitted unimportant aspects but this basic approach is working for us.

          Nick Jones added a comment - - edited Indeed, jglick , what you described is essentially what I ended up doing (in declarative pipeline): environment { GITUSER = credentials( 'some_credential_id' ) } steps { sh 'git tag -a some.tag -m "some message" ' sh 'git push https: //${GITUSER_USR}:${GITUSER_PSW}@some.git.url some.tag' } I've omitted unimportant aspects but this basic approach is working for us.

          Jesse Glick added a comment -

          Yes; the idea would be to allow you to write something along the lines of

          withGit(credentialsId: 'git-login') {
            sh 'git tag whatever && git push origin whatever'
          }

          Jesse Glick added a comment - Yes; the idea would be to allow you to write something along the lines of withGit(credentialsId: 'git-login' ) { sh 'git tag whatever && git push origin whatever' }

          FTR, the workaround I used until there is a better way:

          withCredentials([sshUserPrivateKey(credentialsId: '...', keyFileVariable: 'GITHUB_KEY')]) {
              sh 'echo ssh -i $GITHUB_KEY -l git -o StrictHostKeyChecking=no \\"\\$@\\" > run_ssh.sh'
              sh 'chmod +x run_ssh.sh'
              withEnv(['GIT_SSH=run_ssh.sh']) {
                  sh 'git push origin whatever'
              }
          }
          

          Alexey Sokolov added a comment - FTR, the workaround I used until there is a better way: withCredentials([sshUserPrivateKey(credentialsId: '...' , keyFileVariable: 'GITHUB_KEY' )]) { sh 'echo ssh -i $GITHUB_KEY -l git -o StrictHostKeyChecking=no \\ "\\$@\\" > run_ssh.sh' sh 'chmod +x run_ssh.sh' withEnv([ 'GIT_SSH=run_ssh.sh' ]) { sh 'git push origin whatever' } }

          Andrew Gray added a comment -

          Is this going to be fixed anytime soon?

          Andrew Gray added a comment - Is this going to be fixed anytime soon?

          Paul Skiba added a comment -

          I added this issue as a project idea for GSoC 2018 but I hope that this issue will be fixed before summer. I don't think that a student will take this project but hope dies last 

          Paul Skiba added a comment - I added this issue as a project idea for GSoC 2018  but I hope that this issue will be fixed before summer. I don't think that a student will take this project but hope dies last 

          Andrew Gray added a comment -

          I find it amazing that there are still these massive glaring gaps in functionality (esp for Git) in critical functions all over declarative pipeline.

           

          They're supposed to be the future but not there yet.

          Andrew Gray added a comment - I find it amazing that there are still these massive glaring gaps in functionality (esp for Git) in critical functions all over declarative pipeline.   They're supposed to be the future but not there yet.

          Kyle Wiering added a comment -

          The CLI scripted work-around is functional. That is the problem. apgray.

          Kyle Wiering added a comment - The CLI scripted work-around is functional. That is the problem. apgray .

          Taras Postument added a comment - - edited

          A bit cleaner workaround here:

              withCredentials([sshUserPrivateKey(credentialsId: 'your-github-key-id', keyFileVariable: 'GITHUB_KEY')]) {
                  withEnv(["GIT_SSH_COMMAND=ssh -i $GITHUB_KEY -o StrictHostKeyChecking=no"]) {
                      git clone/push git@...
                  }
              }
          

          Taras Postument added a comment - - edited A bit cleaner workaround here: withCredentials([sshUserPrivateKey(credentialsId: 'your-github-key-id' , keyFileVariable: 'GITHUB_KEY' )]) { withEnv([ "GIT_SSH_COMMAND=ssh -i $GITHUB_KEY -o StrictHostKeyChecking=no" ]) { git clone/push git@... } }

          Its preferable to avoid writing secrets to the workspace. Although, ssh agent could theoretically be accessed from another worker by figuring out the SSH_AUTH_SOCK.

          withEnv(['GIT_SSH_COMMAND=ssh -o StrictHostKeyChecking=no']) {
            sshagent(credentials: ['your-github-key']) {
            ...
            }
          }
          

          Joshua Hoblitt added a comment - Its preferable to avoid writing secrets to the workspace. Although, ssh agent could theoretically be accessed from another worker by figuring out the SSH_AUTH_SOCK . withEnv([ 'GIT_SSH_COMMAND=ssh -o StrictHostKeyChecking=no' ]) { sshagent(credentials: [ 'your-github-key' ]) { ... } }

          Jesse Glick added a comment -

          The sshagent step is more convenient, but it boils down to more or less the same thing. withCredentials does not write files to the workspace per se, but to an @tmp sibling directory.

          Jesse Glick added a comment - The sshagent step is more convenient, but it boils down to more or less the same thing. withCredentials does not write files to the workspace per se, but to an @tmp sibling directory.

          David Taylor added a comment -

          any updates would be appreciated. my team is working on pipelines and we need this plugin to be updated in order to push back to Bitbucket from windows without using ssh keys.

          David Taylor added a comment - any updates would be appreciated. my team is working on pipelines and we need this plugin to be updated in order to push back to Bitbucket from windows without using ssh keys.

          You can consider using https://wiki.jenkins.io/display/JENKINS/Pretested+Integration+Plugin if it matches your needs. If not feel free to propose your missing needs..

          It is using the credentials from the Git SCM plugin and uses it in the "publisher". It supports http/windows ..

          Claus Schneider added a comment - You can consider using https://wiki.jenkins.io/display/JENKINS/Pretested+Integration+Plugin  if it matches your needs. If not feel free to propose your missing needs.. It is using the credentials from the Git SCM plugin and uses it in the "publisher". It supports http/windows ..

          David Taylor added a comment -

          bicschneider thank you for the plugin recommendation. we will look at this and see what works!

          David Taylor added a comment - bicschneider thank you for the plugin recommendation. we will look at this and see what works!

          David Taylor added a comment -

          bicschneider we've looked at the pre-tested plugin and we're not sure if it will fit with what we're trying to do. we really just need a way to push changes back to Bitbucket without using SSH keys from the build machine. The Git Plugin / Git Publisher plugin should allow for this but we're not sure how to implement it yet.

          David Taylor added a comment - bicschneider we've looked at the pre-tested plugin and we're not sure if it will fit with what we're trying to do. we really just need a way to push changes back to Bitbucket without using SSH keys from the build machine. The Git Plugin / Git Publisher plugin should allow for this but we're not sure how to implement it yet.

          Jesse Glick added a comment -

          As already noted in this issue, you can already use withCredentials or sshagent to accomplish this use case. This enhancement would simply make it more convenient.

          Jesse Glick added a comment - As already noted in this issue, you can already use withCredentials or sshagent to accomplish this use case. This enhancement would simply make it more convenient.

          Ruslan Zenin added a comment - - edited

          Hi jglick . would it be possible to provide a sample pipeline script how we can accomplish "git push withCredentials or sshagent"? There is a use case for my team to build projects and change some of the artifacts and then push back to the SCM.
          Thank you in advance!
          Note: I found this example: https://jenkins.io/doc/pipeline/examples/#push-git-repo

          withCredentials([usernamePassword(credentialsId: 'git-pass-credentials-ID', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) { sh("git tag -a some_tag -m 'Jenkins'") sh('git push https://${GIT_USERNAME}:${GIT_PASSWORD}@<REPO> --tags') }

          Ruslan Zenin added a comment - - edited Hi jglick  . would it be possible to provide a sample pipeline script how we can accomplish "git push withCredentials or sshagent" ? There is a use case for my team to build projects and change some of the artifacts and then push back to the SCM. Thank you in advance! Note: I found this example: https://jenkins.io/doc/pipeline/examples/#push-git-repo withCredentials( [usernamePassword(credentialsId: 'git-pass-credentials-ID', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')] ) { sh("git tag -a some_tag -m 'Jenkins'") sh('git push https://$ {GIT_USERNAME}:${GIT_PASSWORD}@<REPO> --tags') }

          Kamil Demecki added a comment -

          zeninr Above there are examples working for me with ssh key. I didn't check how to use credentials with password if you are asking about this case specific.

          sshagent(['credentiald-id-using-ssh-key']) 
           {
              sh('git command or program calling git inside') 
           }
          

          Kamil Demecki added a comment - zeninr Above there are examples working for me with ssh key. I didn't check how to use credentials with password if you are asking about this case specific. sshagent([ 'credentiald-id-using-ssh-key' ]) {     sh( 'git command or program calling git inside' ) }

          Ruslan Zenin added a comment - - edited

          kodstark Thank you! That worked for me!

          Ruslan Zenin added a comment - - edited kodstark Thank you! That worked for me!

          chirs damour added a comment - - edited

          we had been using the sshagent workaround, but recently changed to using the "Checkout over SSH" git scm custom behaviour.  This made the sshagent wrapper unecessary in the scripts (i'm guessing cause it basically wraps your whole pipeline as such..kind of). Not sure how portable this is to windows or non git environments but was easier for us.

          Also to control the commit/author email was used the Custom user name/e-mail address custom behaviour

          chirs damour added a comment - - edited we had been using the sshagent workaround, but recently changed to using the "Checkout over SSH" git scm custom behaviour.  This made the sshagent wrapper unecessary in the scripts (i'm guessing cause it basically wraps your whole pipeline as such..kind of). Not sure how portable this is to windows or non git environments but was easier for us. Also to control the commit/author email was used the  Custom user name/e-mail address  custom behaviour

          Jesper Matthiesen added a comment - - edited

          I'm trying to get a post-build git push --tags to work on windows using user/pass with jenkins credentials outside of pipeline/workflow. The git plugin fetches from a remote private repo this way without problems, but I can't get authentication to work when invoking git through windows batch, or groovy as suggested in https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=258095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-258095 above. Shouldn't git operations through the git client in groovy work, if things like fetch, performed by the git plugin, work?

          When fetching, 'using GIT_ASKPASS to set credentials' is printed in the console, but when I try to tag, it isn't, so maybe that's a hint?

           Failing push:

          using credential dd64d000-6f87-4271-90dd-58babca8bb5e
          > git.exe --version # timeout=10
          > git.exe push origin --tags
          ERROR: Failed to evaluate groovy script.
          hudson.plugins.git.GitException: Command "git.exe push origin --tags" returned status code 128:
          stdout: 
          stderr: remote: Invalid username or password
          fatal: Authentication failed for 'https://name@bitbucket.org/.../...git/' 

          Jesper Matthiesen added a comment - - edited I'm trying to get a post-build git push --tags to work on windows using user/pass with jenkins credentials outside of pipeline/workflow. The git plugin fetches from a remote private repo this way without problems, but I can't get authentication to work when invoking git through windows batch, or groovy as suggested in https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=258095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-258095 above. Shouldn't git operations through the git client in groovy work, if things like fetch, performed by the git plugin, work? When fetching, 'using GIT_ASKPASS to set credentials' is printed in the console, but when I try to tag, it isn't, so maybe that's a hint?  Failing push: using credential dd64d000-6f87-4271-90dd-58babca8bb5e > git.exe --version # timeout=10 > git.exe push origin --tags ERROR: Failed to evaluate groovy script. hudson.plugins.git.GitException: Command "git.exe push origin --tags" returned status code 128: stdout: stderr: remote: Invalid username or password fatal: Authentication failed for 'https: //name@bitbucket.org/.../...git/'

          Nick Jones added a comment -

          matthiesenj, perhaps my answer at https://stackoverflow.com/a/37753202/466874 might help with the authentication specifically? With Git configured as I noted in that answer (basically, disabling the credential.helper), we're using the git CLI successfully on Windows agents to push tags, branches, etc.. After configuring Git this way, the steps we do are 1) bind the Git credentials (to make them available as username/password environment variables), 2) use the https://USERNAME:PASSWORD@URL syntax when pushing anything back to GitHub (e.g., https://foo:bar@github.com/MyRepository).

          Nick Jones added a comment - matthiesenj , perhaps my answer at https://stackoverflow.com/a/37753202/466874 might help with the authentication specifically? With Git configured as I noted in that answer (basically, disabling the credential.helper), we're using the git CLI successfully on Windows agents to push tags, branches, etc.. After configuring Git this way, the steps we do are 1) bind the Git credentials (to make them available as username/password environment variables), 2) use the https://USERNAME:PASSWORD@URL syntax when pushing anything back to GitHub (e.g., https://foo:bar@github.com/MyRepository ).

          medianick I've now made it work by converting the job to pipeline and using the withCredentials directive, thereby injecting the credentials into the url myself. I never found a way to access the credentials to do the same with my previous freestyle project.

          But what I don't understand is why we must do this credential fetching and url massaging ourselves, i.e. why for instance djviking's example using the GitClient class directly doesn't (seem to) work.

          Jesper Matthiesen added a comment - medianick I've now made it work by converting the job to pipeline and using the  withCredentials directive, thereby injecting the credentials into the url myself. I never found a way to access the credentials to do the same with my previous freestyle project. But what I don't understand is why we must do this credential fetching and url massaging ourselves, i.e. why for instance djviking 's example using the GitClient class directly doesn't (seem to) work.

          Michael Beaumont added a comment - - edited

          I think I've come up with a good workaround. We can set the following at the beginning of our build (declarative pipeline in a Github organization):

          sh 'git config --local credential.helper "!p() { echo username=\\$GIT_USERNAME; echo password=\\$GIT_PASSWORD; }; p"'
          

          See git credential helpers

           

          Then, when we want to use the credentials we can use a block like the following: 

          sh 'git tag -m "" ${VERSION_NUMBER}'
          withCredentials([
            usernamePassword(credentialsId: 'github', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD')
          ]) {
            sh 'git push origin ${VERSION_NUMBER}'
          }
          

          This way we don't have to even repeat the URL for the origin remote, which is already set. 

          Michael Beaumont added a comment - - edited I think I've come up with a good workaround. We can set the following at the beginning of our build (declarative pipeline in a Github organization): sh 'git config --local credential.helper "!p() { echo username=\\$GIT_USERNAME; echo password=\\$GIT_PASSWORD; }; p" ' See git credential helpers   Then, when we want to use the credentials we can use a block like the following:  sh 'git tag -m "" ${VERSION_NUMBER}' withCredentials([ usernamePassword(credentialsId: 'github' , usernameVariable: 'GIT_USERNAME' , passwordVariable: 'GIT_PASSWORD' ) ]) { sh 'git push origin ${VERSION_NUMBER}' } This way we don't have to even repeat the URL for the origin remote, which is already set. 

          Kari Niemi added a comment -

          Official coudbees provided instructions for the workaround: https://support.cloudbees.com/hc/en-us/articles/360027646491-Pipeline-Equivalent-to-Git-Publisher

          ...but they have bugs in the examples: withCredentials is used to publish variable GIT_USERNAME  ... and then that variable is referenced with GIT_AUTH_USR in shell step. The same mistake for password variables.

          Kari Niemi added a comment - Official coudbees provided instructions for the workaround: https://support.cloudbees.com/hc/en-us/articles/360027646491-Pipeline-Equivalent-to-Git-Publisher ...but they have bugs in the examples: withCredentials is used to publish variable GIT_USERNAME  ... and then that variable is referenced with GIT_AUTH_USR in shell step. The same mistake for password variables.

          Mark Waite added a comment -

          Git plugin 4.8.0 released July 21, 2021 with the git credentials binding for username / password credentials. Special thanks to Harshit Chopra, the Google Summer of Code 2021 student that implemented it.

          Work continues on the git credentials binding for ssh private key credentials. Planned to be included in a future release of the git plugin.

          Mark Waite added a comment - Git plugin 4.8.0 released July 21, 2021 with the git credentials binding for username / password credentials. Special thanks to Harshit Chopra, the Google Summer of Code 2021 student that implemented it. Work continues on the git credentials binding for ssh private key credentials. Planned to be included in a future release of the git plugin.

          Dee Kryvenko added a comment -

          New gitUsernamePassword binding only accounts for a single set of credentials.

          Often there are different set of credentials required for different git locations.

          Examples would be Go Modules, Terraform Modules and other that are using Git for dependency management. If the current repository we are building have dependencies on other repositories - we need read-write git credentials to the current repository (so we could publish tags, for example) while we need read-only set of credentials to everything else for fetching dependencies.

          If this binding currently implemented as AskPass helper - an easy fix would be to accept additional parameter in the binding that'd tell the location the binding is for.

          Dee Kryvenko added a comment - New gitUsernamePassword binding only accounts for a single set of credentials. Often there are different set of credentials required for different git locations. Examples would be Go Modules, Terraform Modules and other that are using Git for dependency management. If the current repository we are building have dependencies on other repositories - we need read-write git credentials to the current repository (so we could publish tags, for example) while we need read-only set of credentials to everything else for fetching dependencies. If this binding currently implemented as AskPass helper - an easy fix would be to accept additional parameter in the binding that'd tell the location the binding is for.

          Mark Waite added a comment -

          llibicpep couldn't you achieve the same result by having a single credential that is used for read access to all repositories and an additional credential that allows read/write access to the one repository that needs to be written? Read operations (like fetch and clone) would be performed inside a withCredentials block using the read credentials, then a separate withCredentials block would be used to perform the write operations.

          Mark Waite added a comment - llibicpep couldn't you achieve the same result by having a single credential that is used for read access to all repositories and an additional credential that allows read/write access to the one repository that needs to be written? Read operations (like fetch and clone) would be performed inside a withCredentials block using the read credentials, then a separate withCredentials block would be used to perform the write operations.

          Dee Kryvenko added a comment -

          markewaite - you are assuming entire pipeline is a Jenkinsfile. That is far from being even in majority of the cases. In reality - Jenkinsfile delegates to the tools such as Make/Maven/Gradle/NPM/Terraform/etc. User-supplied untrusted code should be able to safely access only what's it allowed to - write to the current repository and read other repositories as dependencies. From the Jenkinsfile level - we don't have exposure into what is going to happen in the user-supplied code. To enable this workflow - we need to pass in to the tool several sets of credentials. And it might be even more complicated as there could be several locations such as GitHub Orgs in play with different sets of credentials each - or even entirely different providers (GitHub+GitLab for example).

          Dee Kryvenko added a comment - markewaite - you are assuming entire pipeline is a Jenkinsfile. That is far from being even in majority of the cases. In reality - Jenkinsfile delegates to the tools such as Make/Maven/Gradle/NPM/Terraform/etc. User-supplied untrusted code should be able to safely access only what's it allowed to - write to the current repository and read other repositories as dependencies. From the Jenkinsfile level - we don't have exposure into what is going to happen in the user-supplied code. To enable this workflow - we need to pass in to the tool several sets of credentials. And it might be even more complicated as there could be several locations such as GitHub Orgs in play with different sets of credentials each - or even entirely different providers (GitHub+GitLab for example).

          Mark Waite added a comment -

          llibicpep since the withCredentials step provides the credential to all git operations inside the sh, bat, or powershell step, I would expect it to be well behaved whether the program that called git was make, maven, gradle, bazel, or any other tool. I think your scenario of mapping different credentials to individual repositories sounds like a more challenging use case than I'm ready to solve using the withCredentials technique.

          Mark Waite added a comment - llibicpep since the withCredentials step provides the credential to all git operations inside the sh , bat , or powershell step, I would expect it to be well behaved whether the program that called git was make, maven, gradle, bazel, or any other tool. I think your scenario of mapping different credentials to individual repositories sounds like a more challenging use case than I'm ready to solve using the withCredentials technique.

          Dee Kryvenko added a comment - - edited

          markewaite the "program" might want to fetch dependencies AND push a release tag, as an example, all in the same time. You don't get to know that from your Jenkinsfile. In the Jenkinsfile it is a single step.

          It's all quite simple actually - the new bindings could accept multiple pairs url:credential-id which would render into a config file like this:

          [credential "https://github.com"]
          	useHttpPath = true
          	helper = ...
          [credential "https://gitlab.com"]
          	useHttpPath = true
          	helper = ...
          

          And the helper then would look into both host AND path in the input (which the latter would only be available when useHttpPath = true) to match with a UsernamePasswordCredentials instance.

          Dee Kryvenko added a comment - - edited markewaite the "program" might want to fetch dependencies AND push a release tag, as an example, all in the same time. You don't get to know that from your Jenkinsfile. In the Jenkinsfile it is a single step. It's all quite simple actually - the new bindings could accept multiple pairs url:credential-id which would render into a config file like this: [credential "https: //github.com" ] useHttpPath = true helper = ... [credential "https: //gitlab.com" ] useHttpPath = true helper = ... And the helper then would look into both host AND path in the input (which the latter would only be available when useHttpPath = true ) to match with a UsernamePasswordCredentials instance.

          David Url added a comment -

          Would this feature work as well in stages that run in containers?

          David Url added a comment - Would this feature work as well in stages that run in containers?

            Unassigned Unassigned
            alecharp Adrien Lecharpentier
            Votes:
            160 Vote for this issue
            Watchers:
            165 Start watching this issue

              Created:
              Updated: