• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • git-plugin
    • None

      We need to make sure that commit SHA that triggered the build via webhook is the same SHA that the repo is being checkout out.

      Use case:

      Dev-1 merges changes to repo 'a'. Within seconds, Dev-2 merges changes to repo 'a'.

      Expectation:

      Each commit triggers a separate build based on the webhook SHA that triggered the build.

      Current result:

      SHA of Jenkinsfile might be not the same as checkout code, since the merges might happen within seconds of each other.

          [JENKINS-59210] Build exactly the commit from the webhook

          Mark Waite added a comment - - edited

          Please provide detailed steps that will allow someone else to duplicate the problem. Based on your description, I can't tell the answers to the following questions (and likely others related to them):

          • Which syntax are you using, declarative or scripted?
          • How is the checkout performed in your Pipeline, as the implicit checkout of a declarative Pipeline or as an explicit checkout from a scripted Pipeline?
          • If using an explicit checkout, what arguments are you providing to the explicit checkout?
          • Are multiple repositories used in your Pipeline?
          • What system is generating the webhook (GitHub, Bitbucket, Gitlab, Gitea, ...)?
          • How does the webhook provide the SHA in its payload (name of variable, etc.)?
          • Is the webhook providing an authentication token as part of its payload?
          • Is your Pipeline definition stored in the git repository or in Jenkins?
          • Is your Pipeline running in a multibranch Pipeline or as a single Pipeline job?
          • Are you expecting that Dev-1 change will be built even if the Jenkins query for the tip of the branch finds the more recent Dev-2 change?

          Please provide more details describing your Pipeline definition and how you see the problem.

          Mark Waite added a comment - - edited Please provide detailed steps that will allow someone else to duplicate the problem. Based on your description, I can't tell the answers to the following questions (and likely others related to them): Which syntax are you using, declarative or scripted? How is the checkout performed in your Pipeline, as the implicit checkout of a declarative Pipeline or as an explicit checkout from a scripted Pipeline? If using an explicit checkout, what arguments are you providing to the explicit checkout? Are multiple repositories used in your Pipeline? What system is generating the webhook (GitHub, Bitbucket, Gitlab, Gitea, ...)? How does the webhook provide the SHA in its payload (name of variable, etc.)? Is the webhook providing an authentication token as part of its payload? Is your Pipeline definition stored in the git repository or in Jenkins? Is your Pipeline running in a multibranch Pipeline or as a single Pipeline job? Are you expecting that Dev-1 change will be built even if the Jenkins query for the tip of the branch finds the more recent Dev-2 change? Please provide more details describing your Pipeline definition and how you see the problem.

          Hi Mark,

          Sorry, for incomplete picture. Here are the details you are asking.:

          • It is a scripted pipeline
          • Explicit checkout from a scripted Pipeline
          • Pipeline snippet:
            • checkout([$class: 'GitSCM',
              branches: [[name: "*/${GIT_BRANCH}"]],
              browser: [$class: 'GithubWeb', repoUrl: "https://${GIT_HOST}/${GIT_OWNER}/${GIT_REPOSITORY}"],
              doGenerateSubmoduleConfigurations: false,
              extensions: [
              [$class: 'RelativeTargetDirectory', relativeTargetDir: "${GIT_REPOSITORY}"],
              [$class: 'LocalBranch', localBranch: "${GIT_BRANCH}"],
              [$class: 'GitLFSPull']
              ],
              submoduleCfg: [],
              userRemoteConfigs: [
              [
              credentialsId: '***********************',
              url: "git@${GIT_HOST}:${GIT_OWNER}/${GIT_REPOSITORY}.git"
              ]
              ]
              ])
          • Enterprise GitHub webhook provider
          • No  authentication token as part of its payload
          • Pipeline definition stored in the git repository
          • Single Pipeline job
          • I am expecting that Dev-1 change will be built separately, even if the Jenkins query for the tip of the branch finds the more recent Dev-2 change.

           

          The actual use case is that our developers provide bug id(s) in commit messages. Once the build is done, we update corresponding bug(s) with the corresponding build information, including:

          • source files changes
          • generated output
          • link to Jenkins build
          • Git SHA of the corresponding commit
          • etc.

          We have a requirement to be able to trigger builds on each commit to insure that only relevant information is inserted into each bug.

          What happens now when commits from 2 developers happen too close to each other, is that once Jenkins queries the specified tip of the branch, it picks up additional changes from Dev-2, which may not be related to changes from Dev-1.

          Ideally, I should be able to find out the SHA of the Jenkinsfile, that triggered the build and use that to do the checkout of the source code.

          Please let me know if I can provide any additional info. 

           

          Oleg Varshavsky added a comment - Hi Mark, Sorry, for incomplete picture. Here are the details you are asking.: It is a scripted pipeline Explicit checkout from a scripted Pipeline Pipeline snippet: checkout([$class: 'GitSCM', branches: [ [name: "*/${GIT_BRANCH}"] ], browser: [$class: 'GithubWeb', repoUrl: "https://${GIT_HOST}/${GIT_OWNER}/${GIT_REPOSITORY}"] , doGenerateSubmoduleConfigurations: false, extensions: [ [$class: 'RelativeTargetDirectory', relativeTargetDir: "${GIT_REPOSITORY}"] , [$class: 'LocalBranch', localBranch: "${GIT_BRANCH}"] , [$class: 'GitLFSPull'] ], submoduleCfg: [], userRemoteConfigs: [ [ credentialsId: '******* ****** **********', url: "git@${GIT_HOST}:${GIT_OWNER}/${GIT_REPOSITORY}.git" ] ] ]) Enterprise GitHub webhook provider No  authentication token as part of its payload Pipeline definition stored in the git repository Single Pipeline job I am expecting that Dev-1 change will be built separately, even if the Jenkins query for the tip of the branch finds the more recent Dev-2 change.   The actual use case is that our developers provide bug id(s) in commit messages. Once the build is done, we update corresponding bug(s) with the corresponding build information, including: source files changes generated output link to Jenkins build Git SHA of the corresponding commit etc. We have a requirement to be able to trigger builds on each commit to insure that only relevant information is inserted into each bug. What happens now when commits from 2 developers happen too close to each other, is that once Jenkins queries the specified tip of the branch, it picks up additional changes from Dev-2, which may not be related to changes from Dev-1. Ideally, I should be able to find out the SHA of the Jenkinsfile, that triggered the build and use that to do the checkout of the source code. Please let me know if I can provide any additional info.   

          Mark Waite added a comment -

          The git plugin is designed to take the most recent commit from the tip of the branch it is monitoring. You want to build a specific commit which may not be the latest commit. There isn't a way to do that directly.

          You could consider creating a parameterized job which takes a SHA1 as an argument. You could give it a default value of HEAD and then invoke it with the SHA1 value that you want to build. You'll need to find some way to get the SHA1 from the repository and assign it as a parameter value when that job is run.

          Mark Waite added a comment - The git plugin is designed to take the most recent commit from the tip of the branch it is monitoring. You want to build a specific commit which may not be the latest commit. There isn't a way to do that directly. You could consider creating a parameterized job which takes a SHA1 as an argument. You could give it a default value of HEAD and then invoke it with the SHA1 value that you want to build. You'll need to find some way to get the SHA1 from the repository and assign it as a parameter value when that job is run.

          Hi Mark,

          In a freestyle Jenkins job this is not an issue, because once the webhook fires of, it checks out the code, which is then ready to be built. However, with a Pipeline job, it is a two step process. It first checks out the Jenkinsfile, then uses it to execute the checkout stage, if defined. 

          Oleg Varshavsky added a comment - Hi Mark, In a freestyle Jenkins job this is not an issue, because once the webhook fires of, it checks out the code, which is then ready to be built. However, with a Pipeline job, it is a two step process. It first checks out the Jenkinsfile, then uses it to execute the checkout stage, if defined. 

          Mark Waite added a comment -

          As far as I understand it, the Pipeline code strives to assure that the same SHA1 that is used to checkout the Jenkinsfile is also used to checkout the repository for the build. I remember testing in some detail that it was behaving as expected a few years ago. I've not performed those detailed tests recently, but I'm not aware of any known cases where the SHA1 that reads the Jenkinsfile is different from the SHA1 of the checkout step.

          Are you certain that the SHA1 of the Jenkinsfile is different than the SHA1 of the repository in the workspace?

          Can you provide a set of steps that confirm the SHA1 of the Jenkinsfile is different than the SHA1 of the repository in the workspace?

          Mark Waite added a comment - As far as I understand it, the Pipeline code strives to assure that the same SHA1 that is used to checkout the Jenkinsfile is also used to checkout the repository for the build. I remember testing in some detail that it was behaving as expected a few years ago. I've not performed those detailed tests recently, but I'm not aware of any known cases where the SHA1 that reads the Jenkinsfile is different from the SHA1 of the checkout step. Are you certain that the SHA1 of the Jenkinsfile is different than the SHA1 of the repository in the workspace? Can you provide a set of steps that confirm the SHA1 of the Jenkinsfile is different than the SHA1 of the repository in the workspace?

          Hi Mark,

          I have attached a screen shot of the Jenkins log, indicating that the SHA of the Jenkinsfile is different from the SHA of the checkout step. That happened because the build was triggered by a commit from Dev-1 (reflected by the first SHA in the screen shot), but within one second, Dev-2 committed their change (reflected by the second SHA in the screen shot).

          Oleg Varshavsky added a comment - Hi Mark, I have attached a screen shot of the Jenkins log, indicating that the SHA of the Jenkinsfile is different from the SHA of the checkout step. That happened because the build was triggered by a commit from Dev-1 (reflected by the first SHA in the screen shot), but within one second, Dev-2 committed their change (reflected by the second SHA in the screen shot).

          Any updates on this? I am sure other people facing similar issues.

          Oleg Varshavsky added a comment - Any updates on this? I am sure other people facing similar issues.

          Mark Waite added a comment -

          No further updates.

          Mark Waite added a comment - No further updates.

            Unassigned Unassigned
            varsho Oleg Varshavsky
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: