Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38674

GitHubCommitStatusSetter unusable in pipeline with multiple checkout steps

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • github-plugin
    • None

      We're using the Shared Libraries Plugin with our Jenkins Pipeline along with a checkout step.

      Combined this means there are two separate checkouts when the pipeline runs.

      When I then try to call a step with GitHubCommitStatusSetter, it then tries to set the status for both commits against both repositories (i.e. 4 different combinations of commit hash/repos). This results in {{FileNotFoundException}}s that fail the build. But this will obviously always fail because each commit is only valid for one of the repositories.

      I see I can hardcode repository and sha values, but unfortunately with the Github Organizational plugin, I don't have access to the sha directly.

      Is there anything that can be done to make this work? Thanks!

          [JENKINS-38674] GitHubCommitStatusSetter unusable in pipeline with multiple checkout steps

          You can use variables and not hardcode

          Kirill Merkushev added a comment - You can use variables and not hardcode

          Marc Sutter added a comment - - edited

          Hi,

          I had the same issue with Shared Library usage and came up with following workaround.

          Note that this code is part of our shared library repo and that we pass the currentBuild variable to the 'updateGithubCommitStatus' function.

           

          def getRepoURL() {
            sh "git config --get remote.origin.url > .git/remote-url"
            return readFile(".git/remote-url").trim()
          }
          
          def getCommitSha() {
            sh "git rev-parse HEAD > .git/current-commit"
            return readFile(".git/current-commit").trim()
          }
          
          def updateGithubCommitStatus(build) {
            // workaround https://issues.jenkins-ci.org/browse/JENKINS-38674
            repoUrl = getRepoURL()
            commitSha = getCommitSha()
          
            step([
              $class: 'GitHubCommitStatusSetter',
              reposSource: [$class: "ManuallyEnteredRepositorySource", url: repoUrl],
              commitShaSource: [$class: "ManuallyEnteredShaSource", sha: commitSha],
              errorHandlers: [[$class: 'ShallowAnyErrorHandler']],
              statusResultSource: [
                $class: 'ConditionalStatusResultSource',
                results: [
                  [$class: 'BetterThanOrEqualBuildResult', result: 'SUCCESS', state: 'SUCCESS', message: build.description],
                  [$class: 'BetterThanOrEqualBuildResult', result: 'FAILURE', state: 'FAILURE', message: build.description],
                  [$class: 'AnyBuildResult', state: 'FAILURE', message: 'Loophole']
                ]
              ]
            ])
          }
          

           

          This works fine for our needs.

          Cheers

          Marc

           

           

          Marc Sutter added a comment - - edited Hi, I had the same issue with Shared Library usage and came up with following workaround. Note that this code is part of our shared library repo and that we pass the currentBuild variable to the 'updateGithubCommitStatus' function.   def getRepoURL() { sh "git config --get remote.origin.url > .git/remote-url" return readFile( ".git/remote-url" ).trim() } def getCommitSha() { sh "git rev-parse HEAD > .git/current-commit" return readFile( ".git/current-commit" ).trim() } def updateGithubCommitStatus(build) { // workaround https://issues.jenkins-ci.org/browse/JENKINS-38674 repoUrl = getRepoURL() commitSha = getCommitSha() step([ $class: 'GitHubCommitStatusSetter' , reposSource: [$class: "ManuallyEnteredRepositorySource" , url: repoUrl], commitShaSource: [$class: "ManuallyEnteredShaSource" , sha: commitSha], errorHandlers: [[$class: 'ShallowAnyErrorHandler' ]], statusResultSource: [ $class: 'ConditionalStatusResultSource' , results: [ [$class: 'BetterThanOrEqualBuildResult' , result: 'SUCCESS' , state: 'SUCCESS' , message: build.description], [$class: 'BetterThanOrEqualBuildResult' , result: 'FAILURE' , state: 'FAILURE' , message: build.description], [$class: 'AnyBuildResult' , state: 'FAILURE' , message: 'Loophole' ] ] ] ]) }   This works fine for our needs. Cheers Marc    

          Thanks, Marc.

          Added your example to plugin's wiki

          Kirill Merkushev added a comment - Thanks, Marc. Added your example to plugin's wiki

          Justin Vallon added a comment -

          This workaround does not work for PR-MERGE builds where the baseline is merged into the PR, creating a new (temporary) commit.  The sha of the new commit is local, and so "git rev-parse HEAD" is a sha that github knows nothing about.

          The scm object contains a buildChooser (scm.buildChooser or scm.extensions.get(BuildChooserSetting.class)) that is a SpecificRevisionBuildChooser, but I cannot get the revision out of that object (field is private and the method to get the revision takes a bunch of arguments).

          I might be able to get "git rev-parse" to work with something like pull/5 or branch, but that seems like it is going to be messy - maybe there is a way to get a solid git ref-spec for the checkout branch.

          The problem with using BuildDataRevisionShaSource is that it seems to choose the SHA that triggered the build, not the SHA of the main repo.  It would be nice if there were a commitShaSource that evaluated to (basically) buildChooser (SpecificRevisionBuildChooser) revision - the sha of the primary repo.

          Justin Vallon added a comment - This workaround does not work for PR-MERGE builds where the baseline is merged into the PR, creating a new (temporary) commit.  The sha of the new commit is local, and so "git rev-parse HEAD" is a sha that github knows nothing about. The scm object contains a buildChooser (scm.buildChooser or scm.extensions.get(BuildChooserSetting.class)) that is a SpecificRevisionBuildChooser, but I cannot get the revision out of that object (field is private and the method to get the revision takes a bunch of arguments). I might be able to get "git rev-parse" to work with something like pull/5 or branch, but that seems like it is going to be messy - maybe there is a way to get a solid git ref-spec for the checkout branch. The problem with using BuildDataRevisionShaSource is that it seems to choose the SHA that triggered the build, not the SHA of the main repo.  It would be nice if there were a commitShaSource that evaluated to (basically) buildChooser (SpecificRevisionBuildChooser) revision - the sha of the primary repo.

          Gourav Pattnaik added a comment - - edited

          lanwen I'm also having issue when I'm checking out multiple repos into different directories. I'm running into this error. This is my pipeline code:

          stage('in-git-invariant')

          { agent

          { label 'integration' }

          steps {
          echo "Running on node: ${env.NODE_NAME}"
          script {
          ['in', 'in-frontend', 'in-solr', 'in-oms', 'in-cms', 'in-kit'].each { repoName ->
          checkoutStep("*/master", "${repoName}", "${repoName}")
          }
          withCredentials([string(credentialsId: 'github-internations-jenkins-composer-token', variable: 'GITHUB_COMPOSER_TOKEN')]) {
          dir('in-kit')

          { phpComposerInstall() }

          sh "php in-kit/bin/in-kit git:invariant ."
          }
          }
          }
          post {
          always

          { updateGithubCommitStatus() }

          }
          }

           

          def checkoutStep(branchname, repo, targetdir = "") {
          checkout changelog: false, poll: false,
          scm: [$class : 'GitSCM',
          branches : [[name: "${branchname}"]],
          browser : [$class: 'GithubWeb', repoUrl: "http://github.com/InterNations/${repo}"],
          doGenerateSubmoduleConfigurations: false,
          extensions : [[$class: 'CleanCheckout'],
          [$class: 'CleanBeforeCheckout'],
          [$class: 'RelativeTargetDirectory', relativeTargetDir: "${targetdir}"],
          [$class: 'CloneOption', depth: 0, noTags: true, reference: '', shallow: false],
          [$class: 'GitLFSPull']],
          submoduleCfg : [],
          userRemoteConfigs : [[credentialsId: 'github-internations-jenkins', url: "git@github.com:InterNations/${repo}.git"]]]
          }

          def updateGithubCommitStatus() {
          repoUrl = getRepoURL()
          commitSha = getCommitSha()
          step([
          $class : 'GitHubCommitStatusSetter',
          reposSource: [$class: "ManuallyEnteredRepositorySource", url: repoUrl],
          commitShaSource: [$class: "ManuallyEnteredShaSource", sha: commitSha],
          //reposSource : [$class: 'AnyDefinedRepositorySource'],
          //commitShaSource : [$class: 'BuildDataRevisionShaSource'],
          contextSource : [$class: 'DefaultCommitContextSource'],
          statusResultSource : [
          $class : 'ConditionalStatusResultSource',
          results: [
          [$class: 'BetterThanOrEqualBuildResult', result: 'UNSTABLE', state: 'SUCCESS'],
          [$class: 'AnyBuildResult', state: 'FAILURE']
          ]
          ],
          statusBackrefSource: [$class: 'BuildRefBackrefSource']
          ])
          }

          ERROR:

           

           
          [2020-09-10T08:47:23.271Z] + git config --get remote.origin.url
          https://ci.internations.org/blue/organizations/jenkins/in-ci-pipeline-dev/detail/in-ci-pipeline-dev/720/pipeline#step-232-log-2[2020-09-10T08:47:23.271Z] /var/lib/jenkins/workspace/in-ci-pipeline-dev@tmp/durable-a61a120f/script.sh: line 1: .git/remote-url: No such file or directory

          Gourav Pattnaik added a comment - - edited lanwen I'm also having issue when I'm checking out multiple repos into different directories. I'm running into this error. This is my pipeline code: stage('in-git-invariant') { agent { label 'integration' } steps { echo "Running on node: ${env.NODE_NAME}" script { ['in', 'in-frontend', 'in-solr', 'in-oms', 'in-cms', 'in-kit'] .each { repoName -> checkoutStep("*/master", "${repoName}", "${repoName}") } withCredentials( [string(credentialsId: 'github-internations-jenkins-composer-token', variable: 'GITHUB_COMPOSER_TOKEN')] ) { dir('in-kit') { phpComposerInstall() } sh "php in-kit/bin/in-kit git:invariant ." } } } post { always { updateGithubCommitStatus() } } }   def checkoutStep(branchname, repo, targetdir = "") { checkout changelog: false, poll: false, scm: [$class : 'GitSCM', branches : [ [name: "${branchname}"] ], browser : [$class: 'GithubWeb', repoUrl: "http://github.com/InterNations/${repo}"] , doGenerateSubmoduleConfigurations: false, extensions : [ [$class: 'CleanCheckout'] , [$class: 'CleanBeforeCheckout'] , [$class: 'RelativeTargetDirectory', relativeTargetDir: "${targetdir}"] , [$class: 'CloneOption', depth: 0, noTags: true, reference: '', shallow: false] , [$class: 'GitLFSPull'] ], submoduleCfg : [], userRemoteConfigs : [ [credentialsId: 'github-internations-jenkins', url: "git@github.com:InterNations/${repo}.git"] ]] } def updateGithubCommitStatus() { repoUrl = getRepoURL() commitSha = getCommitSha() step([ $class : 'GitHubCommitStatusSetter', reposSource: [$class: "ManuallyEnteredRepositorySource", url: repoUrl] , commitShaSource: [$class: "ManuallyEnteredShaSource", sha: commitSha] , //reposSource : [$class: 'AnyDefinedRepositorySource'] , //commitShaSource : [$class: 'BuildDataRevisionShaSource'] , contextSource : [$class: 'DefaultCommitContextSource'] , statusResultSource : [ $class : 'ConditionalStatusResultSource', results: [ [$class: 'BetterThanOrEqualBuildResult', result: 'UNSTABLE', state: 'SUCCESS'] , [$class: 'AnyBuildResult', state: 'FAILURE'] ] ], statusBackrefSource: [$class: 'BuildRefBackrefSource'] ]) } ERROR:     [2020-09-10T08:47:23.271Z] + git config --get remote.origin.url https://ci.internations.org/blue/organizations/jenkins/in-ci-pipeline-dev/detail/in-ci-pipeline-dev/720/pipeline#step-232-log-2 [2020-09-10T08:47:23.271Z] /var/lib/jenkins/workspace/in-ci-pipeline-dev@tmp/durable-a61a120f/script.sh: line 1: .git/remote-url: No such file or directory

          Benjamin Poulain added a comment - - edited

          Hi,

           

          It seems I have the same issue with the official workaround to get the SHA:

          def getCommitSha() {
            sh "git rev-parse HEAD > .git/current-commit"
            return readFile(".git/current-commit").trim()
          }

           

          When there is a PR with a new temporary commit, we got the reference of the temporary SHA which does not exist from GitHub.

          In fact I have this issue in this specific case (the new produced commit):

          Merging remotes/origin/val commit XXXAAAXXX into PR head commit XXXBBBXXX
          Merge succeeded, producing XXXCCCXXX
          Checking out Revision XXXCCCXXX (PR-5)
          Commit message: "Merge commit 'XXXAAAXXX' into HEAD"

           

          I found this other way to get the SHA which is working for us:

          def getCommitSha() {
            sh "git show-ref -s $GIT_BRANCH > .git/current-commit"
            return readFile(".git/current-commit").trim()
          }
          

           

          I hope it will fix your issues

          Ben

           

          Benjamin Poulain added a comment - - edited Hi,   It seems I have the same issue with the official workaround to get the SHA: def getCommitSha() { sh "git rev-parse HEAD > .git/current-commit" return readFile( ".git/current-commit" ).trim() }   When there is a PR with a new temporary commit, we got the reference of the temporary SHA which does not exist from GitHub. In fact I have this issue in this specific case (the new produced commit): Merging remotes/origin/val commit XXXAAAXXX into PR head commit XXXBBBXXX Merge succeeded, producing XXXCCCXXX Checking out Revision XXXCCCXXX (PR-5) Commit message: "Merge commit 'XXXAAAXXX' into HEAD"   I found this other way to get the SHA which is working for us: def getCommitSha() { sh "git show-ref -s $GIT_BRANCH > .git/current-commit" return readFile( ".git/current-commit" ).trim() }   I hope it will fix your issues Ben  

            lanwen Kirill Merkushev
            lsglick Logan Glickfield
            Votes:
            3 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: