Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38533

Checkout the pull request merge refs directly

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • bitbucket-branch-source-plugin
    • None
    • Jenkins LTS v2.60.2, Bitbucket branch source plugin v2.2.2, Bitbucket Server v4.12.1

      Updated whole issue after new version of bitbucket-branch-source plugin

      We use the Bitbucket branch source plugin with a private Bitbucket server instance.
      We create the pull requests of our features branches quite early to see, if the pull request merge can be build based on our Bitbucket pull request merge strategy. So we want to build the pull-requests/*/merge refs.

       

      An example repo, where git ls-remote shows the following refs:
 9e4f3f544c1595d5fa23b0acc6480abe852f8494 HEAD
 dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/heads/feature
 9e4f3f544c1595d5fa23b0acc6480abe852f8494 refs/heads/master
 dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/pull-requests/1/from
 3219b86ea2902d63d983d849f2c95f288587401a refs/pull-requests/1/merge

      The different discover pull request from origin strategies for the pull-requests merge produces always a checkout of refs/pull-requests/1/from and perform a local merge.

      It seems not possible to checkout the merge ref directly. Is there a reason for this behavior and how can can this be configured? Because the local merge is done with the default merge settings and not with the one configured in Bitbucket.

       

        1. Screen Shot 2017-08-29 at 13.58.32.png
          Screen Shot 2017-08-29 at 13.58.32.png
          304 kB

          [JENKINS-38533] Checkout the pull request merge refs directly

          Carsten Kuhl created issue -

          Alexander Gerock added a comment -

          Stash Pull Request Builder Plugin uses Stash REST API to check for changes on pull requests:

          • https://<YOUR-HOST>/stash/rest/api/1.0/projects/<PROJECT>/repos/<REPO>/pull-requests/

          If a new commit is present build job is started

          • last revision is retrieved by
            • git fetch --tags --progress <GIT-REPO-URL> +refs/pull-requests/:refs/remotes/origin/pr/
            • git rev-parse refs/remotes/origin/pr/<PR-ID>/from^ {commit}

          The problem is that after pushing a commit on a branch for wich a pull request exists these changes are retrieved immediately by REST API but refs are not updated so that build job checks out not the latest commit.

          The pull request (refs) is only updated if the following call is made:

          • https://<YOUR-HOST>/stash/rest/api/latest/projects/<PROJECT>/repos/<REPO>/pull-requests/<PR-ID>/merge

          This call is made if you open a pull request in stash web page. This is the official solution:

          Workaround for Stash Pull Request Builder Plugin:

          • Activate "Build only if PR is mergeable" of Stash Pull Requests Builder on Jenkins to enforce REST API call to .../merge

          Possible solution:

          • Always call https://<YOUR-HOST>/stash/rest/api/latest/projects/<PROJECT>/repos/<REPO>/pull-requests/<PR-ID>/merge if there are changes on https://<YOUR-HOST>/stash/rest/api/1.0/projects/<PROJECT>/repos/<REPO>/pull-requests/

          Alexander Gerock added a comment - Stash Pull Request Builder Plugin uses Stash REST API to check for changes on pull requests: https://<YOUR-HOST>/stash/rest/api/1.0/projects/<PROJECT>/repos/<REPO>/pull-requests/ If a new commit is present build job is started last revision is retrieved by git fetch --tags --progress <GIT-REPO-URL> +refs/pull-requests/:refs/remotes/origin/pr/ git rev-parse refs/remotes/origin/pr/<PR-ID>/from^ {commit} The problem is that after pushing a commit on a branch for wich a pull request exists these changes are retrieved immediately by REST API but refs are not updated so that build job checks out not the latest commit. The pull request (refs) is only updated if the following call is made: https://<YOUR-HOST>/stash/rest/api/latest/projects/<PROJECT>/repos/<REPO>/pull-requests/<PR-ID>/merge This call is made if you open a pull request in stash web page. This is the official solution: https://answers.atlassian.com/questions/239988/change-pull-request-refs-after-commit-instead-of-after-approval-or-workaround Workaround for Stash Pull Request Builder Plugin: Activate "Build only if PR is mergeable" of Stash Pull Requests Builder on Jenkins to enforce REST API call to .../merge Possible solution: Always call https://<YOUR-HOST>/stash/rest/api/latest/projects/<PROJECT>/repos/<REPO>/pull-requests/<PR-ID>/merge if there are changes on https://<YOUR-HOST>/stash/rest/api/1.0/projects/<PROJECT>/repos/<REPO>/pull-requests/

          Carsten Kuhl added a comment -

          Hello ee2284b30c,
          thanks for your input, but I think you talked about another plug-in.

          Carsten Kuhl added a comment - Hello ee2284b30c , thanks for your input, but I think you talked about another plug-in.

          Alexander Gerock added a comment -

          Hello bbio13,

          you are right. I meant stash-pullrequest-builder-plugin. Sorry for the wrong comment.

          Alexander Gerock added a comment - Hello bbio13 , you are right. I meant stash-pullrequest-builder-plugin. Sorry for the wrong comment.
          James Dumay made changes -
          Component/s New: stash-pullrequest-builder-plugin [ 20028 ]
          Component/s Original: bitbucket-branch-source-plugin [ 21428 ]
          Carsten Kuhl made changes -
          Component/s New: bitbucket-branch-source-plugin [ 21428 ]
          Component/s Original: stash-pullrequest-builder-plugin [ 20028 ]
          Assignee Original: Antonio Muñiz [ amuniz ] New: Stephen Connolly [ stephenconnolly ]
          Description Original: We use Bitbucket Team/Project jobs for our Bitbucket Server Projects, where we use Jenkinsfiles for the build configurations.
          The build jobs for different branches works fine and as expected, but the pull requests does not.
          *For example*:
          Repository A with two branches (master/feature) and one pull request to merge feature into master.
          A git ls-remote shows the following refs:
          9e4f3f544c1595d5fa23b0acc6480abe852f8494 HEAD
          dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/heads/feature
          9e4f3f544c1595d5fa23b0acc6480abe852f8494 refs/heads/master
          dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/pull-requests/1/from
          3219b86ea2902d63d983d849f2c95f288587401a refs/pull-requests/1/merge

          The build jobs checkouts the following:
          master -> 9e4f3f544c1595d5fa23b0acc6480abe852f8494
          feature -> dac124e3e1a73d2a9a10d7a99ad79d733ada6db3
          pull-request-1 -> {color:red}dac124e3e1a73d2a9a10d7a99ad79d733ada6db3{color}

          In the jenkins log you see the correct pull request branch name:
          Checking out Revision dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 (PR-1)

          But in my opinion the PR jobs should checkout the merge result from the bitbucket (refs/pull-requests/1/merge) and not the revision of the feature. This is also reflected that the build result of the pull request in bitbucket server shows the result from both jobs (feature and pull-request)           		New: {{Updated whole issue after new version of bitbucket-branch-source plugin}}

          We use the Bitbucket branch source plugin with a private Bitbucket server instance.
          We create the pull requests of our features branches quite early to see, if the pull request merge can be build based on our Bitbucket pull request merge strategy. So we want to build the pull-requests/*/merge refs.

           
          {code:java}
          An example repo, where git ls-remote shows the following refs:
           9e4f3f544c1595d5fa23b0acc6480abe852f8494 HEAD
           dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/heads/feature
           9e4f3f544c1595d5fa23b0acc6480abe852f8494 refs/heads/master
           dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/pull-requests/1/from
           3219b86ea2902d63d983d849f2c95f288587401a refs/pull-requests/1/merge
          {code}
          The different _discover pull request from origin_ strategies for the pull-requests merge produces always a checkout of refs/pull-requests/1/from and perform a local merge.

          It seems not possible to checkout the _merge_ ref directly. Is there a reason for this behavior and how can can this be configured? Because the local merge is done with the default merge settings and not with the one configured in Bitbucket.

           
          Environment Original: Jenkins LTS v2.7.4, Bitbucket branch source plugin v1.8, Bitbucket Server v4.9.1 New: Jenkins LTS v2.60.2, Bitbucket branch source plugin v2.2.2, Bitbucket Server v4.12.1
          Issue Type Original: Bug [ 1 ] New: Improvement [ 4 ]
          Priority Original: Major [ 3 ] New: Minor [ 4 ]
          Summary Original: Pull Requests checkout the wrong commit New: Checkout the pull request merge refs directly

          Stephen Connolly added a comment -

          If we checked out refs/pull-requests/1/merge then what would we end up checking out for a non-mergable commit? How would we know the commit was not mergable?

          The semantics of refs/pull-requests/1/merge as I understand it are that the ref is the last mergable merge... it may not reflect the current refs/pull-requests/1/from thus we would think we had successfully built the current refs/pull-requests/1/from and incorrectly report a successful build status...

          Alternatively if Bitbucket removes the refs/pull-requests/1/merge ref when the commit is not mergable, then you will get an invalid ref which doesn't exactly tell you why the ref is missing or hint to the user the difference between trying to build a PR that has now disappeared or trying to build a PR that cannot be merged.

          Finally, the refs/pull-requests/1/from ref is updated more promptly than the refs/pull-requests/1/merge because the former is a simple ref update while the latter needs to actually compute the merge, so event handling would be more complex if we used refs/pull-requests/1/merge

          The only solution we could find was to do the merge ourselves, that lets us report the merge failure as a merge failure...

          Now if you want to be able to customize the merge strategy, that seems reasonable... but not something for the base plugin to expose as a behaviour... rather it would be better if the customization of the behaviour was part of an extension plugin (though we'd need to expose the ability for plugins to customize in the base plugin)

          Stephen Connolly added a comment - If we checked out refs/pull-requests/1/merge then what would we end up checking out for a non-mergable commit? How would we know the commit was not mergable? The semantics of refs/pull-requests/1/merge as I understand it are that the ref is the last mergable merge... it may not reflect the current refs/pull-requests/1/from thus we would think we had successfully built the current refs/pull-requests/1/from and incorrectly report a successful build status... Alternatively if Bitbucket removes the refs/pull-requests/1/merge ref when the commit is not mergable, then you will get an invalid ref which doesn't exactly tell you why the ref is missing or hint to the user the difference between trying to build a PR that has now disappeared or trying to build a PR that cannot be merged. Finally, the refs/pull-requests/1/from ref is updated more promptly than the refs/pull-requests/1/merge because the former is a simple ref update while the latter needs to actually compute the merge, so event handling would be more complex if we used refs/pull-requests/1/merge The only solution we could find was to do the merge ourselves, that lets us report the merge failure as a merge failure... Now if you want to be able to customize the merge strategy, that seems reasonable... but not something for the base plugin to expose as a behaviour... rather it would be better if the customization of the behaviour was part of an extension plugin (though we'd need to expose the ability for plugins to customize in the base plugin)

          Carsten Kuhl added a comment -

          Hello stephenconnolly,

          thanks for your comment and detailed explanations.

          From my experience is the current Bitbucket behavior to delete the merge ref, if the commit is not mergable. So a ls-remote only shows the from and not the merge ref for a PR. The answer why is something not mergable can certainly only be answered by Bitbucket. For example the default merge is valid and Bitbucket denies the merge because of the --ff-only option.

          So if a build is triggered for the merge and a check or checkout itself reports a missing merge ref, the user have to look at Bitbucket for further information.

          For the other problem of the lazy update of Bitbucket PR merge ref, this is indeed a problem. I've seen the case, where an update of the from triggers a build via webhook, where the merge ref still points to the old SHA. I don't know how this is solved with other Git repro hosting services like Github, but I think these should be solved at Bitbucket side. So the webhook should trigger a PR build for the from ref, if the from ref is updated and for the merge ref, if the merge ref is updated.

          Is this possible with the current webhook api?

          Carsten Kuhl added a comment - Hello stephenconnolly , thanks for your comment and detailed explanations. From my experience is the current Bitbucket behavior to delete the merge ref, if the commit is not mergable. So a ls-remote only shows the from and not the merge ref for a PR. The answer why is something not mergable can certainly only be answered by Bitbucket. For example the default merge is valid and Bitbucket denies the merge because of the --ff-only option. So if a build is triggered for the merge and a check or checkout itself reports a missing merge ref, the user have to look at Bitbucket for further information. For the other problem of the lazy update of Bitbucket PR merge ref, this is indeed a problem. I've seen the case, where an update of the from triggers a build via webhook, where the merge ref still points to the old SHA. I don't know how this is solved with other Git repro hosting services like Github, but I think these should be solved at Bitbucket side. So the webhook should trigger a PR build for the from ref, if the from ref is updated and for the merge ref, if the merge ref is updated. Is this possible with the current webhook api?

          Stephen Connolly added a comment -

          Is this possible with the current webhook api?

          AIUI, this is not possible.

          What is probably more productive is to identify the changes to the current merge strategy that would result in a more equivalent merge to that generated by Bitbucket. (right now the only difference I see is the commit message and commit author, which is effectively no difference, but if you can show a case where there is a concrete difference I would view that as a bug that should be fixed)

          Stephen Connolly added a comment - Is this possible with the current webhook api? AIUI, this is not possible. What is probably more productive is to identify the changes to the current merge strategy that would result in a more equivalent merge to that generated by Bitbucket. (right now the only difference I see is the commit message and commit author, which is effectively no difference, but if you can show a case where there is a concrete difference I would view that as a bug that should be fixed)
          Stephen Connolly made changes -
          Attachment New: Screen Shot 2017-08-29 at 13.58.32.png [ 39527 ]

            Unassigned Unassigned
            bbio13 Carsten Kuhl
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: