Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54285

Changeset not reflective of Pull Request

    XMLWordPrintable

Details

    Description

      I'm using Blue Ocean's default multi-branch setup for GitHub. I am using Jenkins to build and test Pull Requests from GitHub.

      I have a repository with multiple subproject directories that I would like to selectively build and test depending on changes in the Pull Request. Looking at the documentation it seems like I should use `when { changeset 'my-dir/**' }` to do this; however the changeset does not contain the changes from the Pull Request, but instead contains the changes since last build + the changes merged to master since the last build.

       

      I think the behavior here should be to consider the changes in the Pull Request to make it possible to use this conditional.

      Attachments

        Activity

          gauthierm Michael Gauthier added a comment - - edited

          I was able to work around this issue by defining a function for my pipeline:

          def boolean hasChangesIn(String module) {
            return !env.CHANGE_TARGET || sh(
              returnStatus: true,
              script: "git diff --name-only origin/${env.CHANGE_TARGET}...${env.GIT_COMMIT} | grep ^${module}/"
            ) == 0
          }
          

          and then using:

          when {
            expression {
              return hasChangesIn('my-dir')
            }
          }
          

          in my pipeline stages

          gauthierm Michael Gauthier added a comment - - edited I was able to work around this issue by defining a function for my pipeline: def boolean hasChangesIn( String module) { return !env.CHANGE_TARGET || sh( returnStatus: true , script: "git diff --name-only origin/${env.CHANGE_TARGET}...${env.GIT_COMMIT} | grep ^${module}/" ) == 0 } and then using: when { expression { return hasChangesIn( 'my-dir' ) } } in my pipeline stages

          After a bit more testing, there's an improved version of the hasChangesIn that works for our monorepo:

          def boolean hasChangesIn(String module) {
              if (env.CHANGE_TARGET == null) {
                  return true;
              }
          
              def MASTER = sh(
                  returnStdout: true,
                  script: "git rev-parse origin/${env.CHANGE_TARGET}"
              ).trim()
          
              // Gets commit hash of HEAD commit. Jenkins will try to merge master into
              // HEAD before running checks. If this is a fast-forward merge, HEAD does
              // not change. If it is not a fast-forward merge, a new commit becomes HEAD
              // so we check for the non-master parent commit hash to get the original
              // HEAD. Jenkins does not save this hash in an environment variable.
              def HEAD = sh(
                  returnStdout: true,
                  script: "git show -s --no-abbrev-commit --pretty=format:%P%n%H%n HEAD | tr ' ' '\n' | grep -v ${MASTER} | head -n 1"
              ).trim()
          
              return sh(
                  returnStatus: true,
                  script: "git diff --name-only ${MASTER}...${HEAD} | grep ^${module}/"
              ) == 0
          }
          
          gauthierm Michael Gauthier added a comment - After a bit more testing, there's an improved version of the hasChangesIn that works for our monorepo: def boolean hasChangesIn( String module) { if (env.CHANGE_TARGET == null ) { return true ; } def MASTER = sh( returnStdout: true , script: "git rev-parse origin/${env.CHANGE_TARGET}" ).trim() // Gets commit hash of HEAD commit. Jenkins will try to merge master into // HEAD before running checks. If this is a fast-forward merge, HEAD does // not change. If it is not a fast-forward merge, a new commit becomes HEAD // so we check for the non-master parent commit hash to get the original // HEAD. Jenkins does not save this hash in an environment variable. def HEAD = sh( returnStdout: true , script: "git show -s --no-abbrev-commit --pretty=format:%P%n%H%n HEAD | tr ' ' '\n' | grep -v ${MASTER} | head -n 1" ).trim() return sh( returnStatus: true , script: "git diff --name-only ${MASTER}...${HEAD} | grep ^${module}/" ) == 0 }

          I wanted to use 'when { changeset ... }' to test changes in my monorepo as well (I'm using Bitbucket branch source, Bitbucket Server) and then came to find this... Why don't we have pull request changes? Webhook payload has the destination branch, so why not have the behaviour the author suggested in the description?

          atikhonova Anna Tikhonova added a comment - I wanted to use 'when { changeset ... }' to test changes in my monorepo as well (I'm using Bitbucket branch source, Bitbucket Server) and then came to find this... Why don't we have pull request changes? Webhook payload has the destination branch, so why not have the behaviour the author suggested in the description?
          cncult Chris Nelson added a comment -

          FWIW, I use this janky workaround to deal with empty changesets on the first PR build

          options {
              // don't let the implicit checkout happem
              skipDefaultCheckout true
          }
          stages {
              stage ('Fix Changelog') {
                  // only do this if there is no prior build
                  when { expression { return !currentBuild.previousBuild } }
                  steps {
                      checkout([
                          $class: 'GitSCM',
                          branches: scm.branches,
                          userRemoteConfigs: scm.userRemoteConfigs,
                          browser: scm.browser,
                          // this extension builds the changesets from the compareTarget branch
                          // Using a variable here, but do what's appropriate for your env
                          extensions: [[$class: 'ChangelogToBranch', options: [compareRemote: 'origin', compareTarget: env.RELEASE_BRANCH]]]
                      ])
                  }
              }
              // no do the normally configured checkout to ensure all configured extensions runs
              // and to generate the changeset for later builds, when jenkins does the right thing
              stage ('Checkout') {
                  steps {
                      checkout scm
                  }
              }
          
          cncult Chris Nelson added a comment - FWIW, I use this janky workaround to deal with empty changesets on the first PR build options { // don't let the implicit checkout happem skipDefaultCheckout true } stages { stage ( 'Fix Changelog' ) { // only do this if there is no prior build when { expression { return !currentBuild.previousBuild } } steps { checkout([ $class: 'GitSCM' , branches: scm.branches, userRemoteConfigs: scm.userRemoteConfigs, browser: scm.browser, // this extension builds the changesets from the compareTarget branch // Using a variable here, but do what's appropriate for your env extensions: [[$class: 'ChangelogToBranch' , options: [compareRemote: 'origin' , compareTarget: env.RELEASE_BRANCH]]] ]) } } // no do the normally configured checkout to ensure all configured extensions runs // and to generate the changeset for later builds, when jenkins does the right thing stage ( 'Checkout' ) { steps { checkout scm } }

          Just discovered this behavior as our team is moving to use PR's with Bitbucket, and one of the teammates pointed out that a few stages which depend on the `changeSet` directive do not happen when building the PR.

          cncult, I like your interim solution. I was wondering why you have the `skipDefaultCheckout` in the pipeline though, didn't you mean to have it as an option for the "Fix Changelog" stage specifically?

          Thanks,

           

          Tsvi

          tsvi Tsvi Mostovicz added a comment - Just discovered this behavior as our team is moving to use PR's with Bitbucket, and one of the teammates pointed out that a few stages which depend on the `changeSet` directive do not happen when building the PR. cncult , I like your interim solution. I was wondering why you have the `skipDefaultCheckout` in the pipeline though, didn't you mean to have it as an option for the "Fix Changelog" stage specifically? Thanks,   Tsvi
          pjdarton pjdarton added a comment - - edited

          I've just discovered this behaviour too, and (using Chris' code as inspiration) I came up with the following:

          def scmToBuildWith=null
          pipeline {
            ...
          
            options {
              ...
              skipDefaultCheckout true // Because we do this in a stage
            }
          
            stages {
              stage ('Calculate Changelog') {
                when { // we are a PR branch's 1st build
                  not { environment name: 'CHANGE_TARGET', value: '' }
                  expression { return !currentBuild.previousBuild }
                }
                steps { // then calculate the changeset as being from the PR's target branch.
                  script {
                    scmToBuildWith=scm
                    def targetBranchForPR = env.CHANGE_TARGET
                    hudson.util.DescribableList scmExtensions=scmToBuildWith.getExtensions()
                    def githubReposWeFetchFrom = scmToBuildWith.userRemoteConfigs
                    def firstRepoWeFetchFrom = githubReposWeFetchFrom?githubReposWeFetchFrom[0]:null
                    def probablyCalledOrigin = firstRepoWeFetchFrom?firstRepoWeFetchFrom.name:'origin'
                    println "Calculating changelog against ${probablyCalledOrigin}/${targetBranchForPR}"
                    def changeLogCalc = new hudson.plugins.git.extensions.impl.ChangelogToBranch(new hudson.plugins.git.ChangelogToBranchOptions(probablyCalledOrigin, targetBranchForPR))
                    scmExtensions.replace(changeLogCalc)
                  }
                }
              }
              // now do the normally configured checkout to ensure all configured extensions runs
              stage ('Checkout') {
                steps {
                  checkout scmToBuildWith
                }
              }
              ...
          

          This results in the official changes for a PR build to be the differences between the PR and the main branch.

          You'll probably need an admin to "approve" some script functions too:

          method hudson.util.DescribableList replace hudson.model.Describable
          new hudson.plugins.git.ChangelogToBranchOptions java.lang.String java.lang.String
          new hudson.plugins.git.extensions.impl.ChangelogToBranch hudson.plugins.git.ChangelogToBranchOptions
          

          To be honest, this is the kind of functionality I'd expect to have been built into the github multi-branch source plugin so that it wasn't necessary to reconfigure the SCM stuff "on the fly" like this, but at least pipelines make this sort of workaround possible...

          PS. It's necessary to make a copy of scm and use that thereafter because every reference to it results in a fresh instance containing only the original data - no changes "stick".

          pjdarton pjdarton added a comment - - edited I've just discovered this behaviour too, and (using Chris' code as inspiration) I came up with the following: def scmToBuildWith= null pipeline { ...  options { ... skipDefaultCheckout true // Because we do this in a stage } stages { stage ( 'Calculate Changelog' ) { when { // we are a PR branch's 1st build not { environment name: 'CHANGE_TARGET' , value: '' }         expression { return !currentBuild.previousBuild }      } steps { // then calculate the changeset as being from the PR's target branch. script {         scmToBuildWith=scm           def targetBranchForPR = env.CHANGE_TARGET           hudson.util.DescribableList scmExtensions=scmToBuildWith.getExtensions()           def githubReposWeFetchFrom = scmToBuildWith.userRemoteConfigs           def firstRepoWeFetchFrom = githubReposWeFetchFrom?githubReposWeFetchFrom[0]: null           def probablyCalledOrigin = firstRepoWeFetchFrom?firstRepoWeFetchFrom.name: 'origin'           println "Calculating changelog against ${probablyCalledOrigin}/${targetBranchForPR}"           def changeLogCalc = new hudson.plugins.git.extensions.impl.ChangelogToBranch( new hudson.plugins.git.ChangelogToBranchOptions(probablyCalledOrigin, targetBranchForPR))           scmExtensions.replace(changeLogCalc)        } } } // now do the normally configured checkout to ensure all configured extensions runs stage ( 'Checkout' ) { steps { checkout scmToBuildWith } } ... This results in the official changes for a PR build to be the differences between the PR and the main branch. You'll probably need an admin to "approve" some script functions too: method hudson.util.DescribableList replace hudson.model.Describable new hudson.plugins.git.ChangelogToBranchOptions java.lang.String java.lang.String new hudson.plugins.git.extensions.impl.ChangelogToBranch hudson.plugins.git.ChangelogToBranchOptions To be honest, this is the kind of functionality I'd expect to have been built into the github multi-branch source plugin so that it wasn't necessary to reconfigure the SCM stuff "on the fly" like this, but at least pipelines make this sort of workaround possible... PS. It's necessary to make a copy of scm and use that thereafter because every reference to it results in a fresh instance containing only the original data - no changes "stick".

          People

            Unassigned Unassigned
            gauthierm Michael Gauthier
            Votes:
            12 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated: