Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34931

Build merge commit instead of PR head

    XMLWordPrintable

Details

    Description

      Add a configuration option (active by default maybe) to build an artificial merge commit generated by merging the PR head commit and the target branch commit.

      Attachments

        Issue Links

          Activity

            abayer Andrew Bayer added a comment -

            There's interest in being able to have webhook-triggered non-PR builds build the specific commit from the event rather than HEAD as well.

            abayer Andrew Bayer added a comment - There's interest in being able to have webhook-triggered non-PR builds build the specific commit from the event rather than HEAD as well.
            dodoent Nenad Miksa added a comment -

            We also need this feature. Currently we have a workaround which creates a local merge commit if env.BRANCH_NAME.startsWith("PR-"). It would be better if this is done automatically.

            dodoent Nenad Miksa added a comment - We also need this feature. Currently we have a workaround which creates a local merge commit if env.BRANCH_NAME.startsWith("PR-"). It would be better if this is done automatically.
            kevin_smets Kevin Smets added a comment -

            dodoent how can you get the target branch to merge with? Haven't found a solution for that so I would be happy to use your workaround for the time being .

            kevin_smets Kevin Smets added a comment - dodoent how can you get the target branch to merge with? Haven't found a solution for that so I would be happy to use your workaround for the time being .
            dodoent Nenad Miksa added a comment -

            kevin_smets, its kind of hardcoded with forward compatibility with current bug in bitbucket branch source plugin :

            // hardcoded global variable
            def targetBranch = "master"
            
            // then later in node before doing a checkout:
            if(env.CHANGE_TARGET != null) {
                targetBranch = env.CHANGE_TARGET
            }
            

            Here is the entire workaround, which uses merge before build when building pull request and also using different fastforward strategies based on global nonFFBranches set which is hardcoded in Jenkinsfile (it would be nice to be able to configure that in plugin settings):

            node {
                        if(env.BRANCH_NAME.startsWith("PR-")) {
                            if(env.CHANGE_TARGET != null) {
                                targetBranch = env.CHANGE_TARGET
                            }
                            echo "Source branches: ${scm.branches[0].name}"
            
                            def ffMode = 'FF_ONLY'
                            if(nonFFBranches.contains(scm.branches[0].name)) {
                                ffMode = 'NO_FF'
                            }
                            // do a full clone with merge before build when building pull request
                            checkout([
                                $class: 'GitSCM',
                                branches: scm.branches,
                                doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
                                extensions: scm.extensions + [[$class: 'CloneOption', noTags: false, reference: '', shallow: false, depth: 0, timeout: 60], [$class: 'PruneStaleBranch'], [$class: 'CheckoutOption', timeout: 60], [$class: 'PreBuildMerge', options: [mergeRemote: 'origin', mergeTarget: targetBranch, fastForwardMode: ffMode]]],
                                submoduleCfg: [],
                                userRemoteConfigs: scm.userRemoteConfigs,
                                browser: [$class: 'BitbucketWeb', repoUrl: 'https://bitbucket.org/microblink/core']
                              ])
                        } else {
                            // do a shallow clone when building feature branch HEAD
                            checkout([
                                $class: 'GitSCM',
                                branches: scm.branches,
                                doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
                                extensions: scm.extensions + [[$class: 'CloneOption', noTags: false, reference: '', shallow: true, depth: 1, timeout: 60], [$class: 'PruneStaleBranch'], [$class: 'CheckoutOption', timeout: 60]],
                                submoduleCfg: [],
                                userRemoteConfigs: scm.userRemoteConfigs,
                                browser: [$class: 'BitbucketWeb', repoUrl: 'https://bitbucket.org/microblink/core']
                              ])
                        }
            }
            

            This workaround requires approval of certain methods in In-Process Script Approval Jenkins settings (namely, access to scm variable getters).

            dodoent Nenad Miksa added a comment - kevin_smets , its kind of hardcoded with forward compatibility with current bug in bitbucket branch source plugin : // hardcoded global variable def targetBranch = "master" // then later in node before doing a checkout: if (env.CHANGE_TARGET != null ) { targetBranch = env.CHANGE_TARGET } Here is the entire workaround, which uses merge before build when building pull request and also using different fastforward strategies based on global nonFFBranches set which is hardcoded in Jenkinsfile (it would be nice to be able to configure that in plugin settings): node { if (env.BRANCH_NAME.startsWith( "PR-" )) { if (env.CHANGE_TARGET != null ) { targetBranch = env.CHANGE_TARGET } echo "Source branches: ${scm.branches[0].name}" def ffMode = 'FF_ONLY' if (nonFFBranches.contains(scm.branches[0].name)) { ffMode = 'NO_FF' } // do a full clone with merge before build when building pull request checkout([ $class: 'GitSCM' , branches: scm.branches, doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations, extensions: scm.extensions + [[$class: 'CloneOption' , noTags: false , reference: '', shallow: false , depth: 0, timeout: 60], [$class: ' PruneStaleBranch '], [$class: ' CheckoutOption ', timeout: 60], [$class: ' PreBuildMerge ', options: [mergeRemote: ' origin', mergeTarget: targetBranch, fastForwardMode: ffMode]]], submoduleCfg: [], userRemoteConfigs: scm.userRemoteConfigs, browser: [$class: 'BitbucketWeb' , repoUrl: 'https: //bitbucket.org/microblink/core' ] ]) } else { // do a shallow clone when building feature branch HEAD checkout([ $class: 'GitSCM' , branches: scm.branches, doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations, extensions: scm.extensions + [[$class: 'CloneOption' , noTags: false , reference: '', shallow: true , depth: 1, timeout: 60], [$class: ' PruneStaleBranch '], [$class: ' CheckoutOption', timeout: 60]], submoduleCfg: [], userRemoteConfigs: scm.userRemoteConfigs, browser: [$class: 'BitbucketWeb' , repoUrl: 'https: //bitbucket.org/microblink/core' ] ]) } } This workaround requires approval of certain methods in In-Process Script Approval Jenkins settings (namely, access to scm variable getters).
            dodoent Nenad Miksa added a comment -

            kevin_smets, we have updated our workaround to work correctly, i.e. it can now resolve the PR target. We use curl and jsawk with Bitbucket API to get info about pull request in question. Here is the code:

            def getPullRequestID() {
                return env.BRANCH_NAME.substring(3)
            }
            
            def obtainBitbucketToken() {
                try {
                    echo "Obtaining BitBucket Access Token"
                    sh "curl -s -X POST https://bitbucket.org/site/oauth2/access_token -u \"${getBitbucketOAuthKey()}:${getBitbucketOAuthSecret()}\" -d grant_type=client_credentials | jsawk 'return this.access_token' | tr -d \"\\n\" > accessToken.txt"
            
                    def accessToken = readFile 'accessToken.txt'
            
                    echo "BitBucket Token request response: ${accessToken}"
            
                    return accessToken
                } catch (error) {
                    echo "Failed to obtain BitBucket access token!"
                    return null
                }
            }
            
            def getPullRequestTargetAndSourceCommit(String accessToken, String repository, String pullRequestID) {
                try {
                    sh "curl -s -X GET https://api.bitbucket.org/2.0/repositories/microblink/${repository}/pullrequests/${pullRequestID} -H \"Authorization: Bearer ${accessToken}\" > pullRequestInfo.json"
                    sh "cat pullRequestInfo.json | jsawk 'return this.source.commit.hash' | tr -d '\\n' > commitID.txt"
                    sh "cat pullRequestInfo.json | jsawk 'return this.destination.branch.name' | tr -d '\\n' > destinationBranch.txt"
                    return [commitHash: readFile('commitID.txt'), destinationBranch: readFile('destinationBranch.txt')]
                } catch (error) {
                    echo "Failed to obtain pull request target and source commit. Reason: ${error}"
                    return null
                }
            }
            

            This can be used like this:

            def accessToken = obtainBitbucketToken()
            def pullRequestID = getPullRequestID()
            def prInfo = getPullRequestTargetAndSourceCommit(accessToken, repository, pullRequestID)
            
            def targetBranch = 'master'
            if (prInfo != null) {
                targetBranch = prInfo.destinationBranch
            }
            

            This code needs to be executed on unix node which has curl and jsawk tools installed. We tried parsing JSON with Groovy's builtin JsonSlurper, but it kept crashing with serialization errors even when used entirely from within function marked with @NonCPS.

            The getPullRequestTargetAndSourceCommit method also returns commit hash which can be used to notify build status to BitBucket, as described in this comment: https://issues.jenkins-ci.org/browse/JENKINS-33507?focusedCommentId=261926&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-261926

            By using curl to directly communicate with BitBucket API we have additional flexibility, like support for approving/unapproving pull requests and much more.

            dodoent Nenad Miksa added a comment - kevin_smets , we have updated our workaround to work correctly, i.e. it can now resolve the PR target. We use curl and jsawk with Bitbucket API to get info about pull request in question. Here is the code: def getPullRequestID() { return env.BRANCH_NAME.substring(3) } def obtainBitbucketToken() { try { echo "Obtaining BitBucket Access Token" sh "curl -s -X POST https: //bitbucket.org/site/oauth2/access_token -u \" ${getBitbucketOAuthKey()}:${getBitbucketOAuthSecret()}\ " -d grant_type=client_credentials | jsawk ' return this .access_token' | tr -d \" \\n\ " > accessToken.txt" def accessToken = readFile 'accessToken.txt' echo "BitBucket Token request response: ${accessToken}" return accessToken } catch (error) { echo "Failed to obtain BitBucket access token!" return null } } def getPullRequestTargetAndSourceCommit( String accessToken, String repository, String pullRequestID) { try { sh "curl -s -X GET https: //api.bitbucket.org/2.0/repositories/microblink/${repository}/pullrequests/${pullRequestID} -H \" Authorization: Bearer ${accessToken}\ " > pullRequestInfo.json" sh "cat pullRequestInfo.json | jsawk ' return this .source.commit.hash' | tr -d '\\n' > commitID.txt" sh "cat pullRequestInfo.json | jsawk ' return this .destination.branch.name' | tr -d '\\n' > destinationBranch.txt" return [commitHash: readFile( 'commitID.txt' ), destinationBranch: readFile( 'destinationBranch.txt' )] } catch (error) { echo "Failed to obtain pull request target and source commit. Reason: ${error}" return null } } This can be used like this: def accessToken = obtainBitbucketToken() def pullRequestID = getPullRequestID() def prInfo = getPullRequestTargetAndSourceCommit(accessToken, repository, pullRequestID) def targetBranch = 'master' if (prInfo != null ) { targetBranch = prInfo.destinationBranch } This code needs to be executed on unix node which has curl and jsawk tools installed. We tried parsing JSON with Groovy's builtin JsonSlurper, but it kept crashing with serialization errors even when used entirely from within function marked with @NonCPS. The getPullRequestTargetAndSourceCommit method also returns commit hash which can be used to notify build status to BitBucket, as described in this comment: https://issues.jenkins-ci.org/browse/JENKINS-33507?focusedCommentId=261926&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-261926 By using curl to directly communicate with BitBucket API we have additional flexibility, like support for approving/unapproving pull requests and much more.

            Is this a duplicate of JENKINS-36283 ?

            nickbrown Nicholas Brown added a comment - Is this a duplicate of JENKINS-36283 ?

            This is a subset of JENKINS-36283. Let's keep it around for now.

            amuniz Antonio Muñiz added a comment - This is a subset of JENKINS-36283 . Let's keep it around for now.
            kevin_smets Kevin Smets added a comment -

            dodoent thank you very much for posting that! Looking into implementing this finally .

            kevin_smets Kevin Smets added a comment - dodoent thank you very much for posting that! Looking into implementing this finally .

            People

              amuniz Antonio Muñiz
              amuniz Antonio Muñiz
              Votes:
              6 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: