• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • None
    • Jenkins LTS v2.60.2, Bitbucket branch source plugin v2.2.2, Bitbucket Server v4.12.1

      Updated whole issue after new version of bitbucket-branch-source plugin

      We use the Bitbucket branch source plugin with a private Bitbucket server instance.
      We create the pull requests of our features branches quite early to see, if the pull request merge can be build based on our Bitbucket pull request merge strategy. So we want to build the pull-requests/*/merge refs.

       

      An example repo, where git ls-remote shows the following refs:
       9e4f3f544c1595d5fa23b0acc6480abe852f8494 HEAD
       dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/heads/feature
       9e4f3f544c1595d5fa23b0acc6480abe852f8494 refs/heads/master
       dac124e3e1a73d2a9a10d7a99ad79d733ada6db3 refs/pull-requests/1/from
       3219b86ea2902d63d983d849f2c95f288587401a refs/pull-requests/1/merge
      

      The different discover pull request from origin strategies for the pull-requests merge produces always a checkout of refs/pull-requests/1/from and perform a local merge.

      It seems not possible to checkout the merge ref directly. Is there a reason for this behavior and how can can this be configured? Because the local merge is done with the default merge settings and not with the one configured in Bitbucket.

       

          [JENKINS-38533] Checkout the pull request merge refs directly

          Hello bbio13,

          you are right. I meant stash-pullrequest-builder-plugin. Sorry for the wrong comment.

          Alexander Gerock added a comment - Hello bbio13 , you are right. I meant stash-pullrequest-builder-plugin. Sorry for the wrong comment.

          If we checked out refs/pull-requests/1/merge then what would we end up checking out for a non-mergable commit? How would we know the commit was not mergable?

          The semantics of refs/pull-requests/1/merge as I understand it are that the ref is the last mergable merge... it may not reflect the current refs/pull-requests/1/from thus we would think we had successfully built the current refs/pull-requests/1/from and incorrectly report a successful build status...

          Alternatively if Bitbucket removes the refs/pull-requests/1/merge ref when the commit is not mergable, then you will get an invalid ref which doesn't exactly tell you why the ref is missing or hint to the user the difference between trying to build a PR that has now disappeared or trying to build a PR that cannot be merged.

          Finally, the refs/pull-requests/1/from ref is updated more promptly than the refs/pull-requests/1/merge because the former is a simple ref update while the latter needs to actually compute the merge, so event handling would be more complex if we used refs/pull-requests/1/merge

          The only solution we could find was to do the merge ourselves, that lets us report the merge failure as a merge failure...

          Now if you want to be able to customize the merge strategy, that seems reasonable... but not something for the base plugin to expose as a behaviour... rather it would be better if the customization of the behaviour was part of an extension plugin (though we'd need to expose the ability for plugins to customize in the base plugin)

          Stephen Connolly added a comment - If we checked out refs/pull-requests/1/merge then what would we end up checking out for a non-mergable commit? How would we know the commit was not mergable? The semantics of refs/pull-requests/1/merge as I understand it are that the ref is the last mergable merge... it may not reflect the current refs/pull-requests/1/from thus we would think we had successfully built the current refs/pull-requests/1/from and incorrectly report a successful build status... Alternatively if Bitbucket removes the refs/pull-requests/1/merge ref when the commit is not mergable, then you will get an invalid ref which doesn't exactly tell you why the ref is missing or hint to the user the difference between trying to build a PR that has now disappeared or trying to build a PR that cannot be merged. Finally, the refs/pull-requests/1/from ref is updated more promptly than the refs/pull-requests/1/merge because the former is a simple ref update while the latter needs to actually compute the merge, so event handling would be more complex if we used refs/pull-requests/1/merge The only solution we could find was to do the merge ourselves, that lets us report the merge failure as a merge failure... Now if you want to be able to customize the merge strategy, that seems reasonable... but not something for the base plugin to expose as a behaviour... rather it would be better if the customization of the behaviour was part of an extension plugin (though we'd need to expose the ability for plugins to customize in the base plugin)

          Carsten Kuhl added a comment -

          Hello stephenconnolly,

          thanks for your comment and detailed explanations.

          From my experience is the current Bitbucket behavior to delete the merge ref, if the commit is not mergable. So a ls-remote only shows the from and not the merge ref for a PR. The answer why is something not mergable can certainly only be answered by Bitbucket. For example the default merge is valid and Bitbucket denies the merge because of the --ff-only option.

          So if a build is triggered for the merge and a check or checkout itself reports a missing merge ref, the user have to look at Bitbucket for further information.

          For the other problem of the lazy update of Bitbucket PR merge ref, this is indeed a problem. I've seen the case, where an update of the from triggers a build via webhook, where the merge ref still points to the old SHA. I don't know how this is solved with other Git repro hosting services like Github, but I think these should be solved at Bitbucket side. So the webhook should trigger a PR build for the from ref, if the from ref is updated and for the merge ref, if the merge ref is updated.

          Is this possible with the current webhook api?

          Carsten Kuhl added a comment - Hello stephenconnolly , thanks for your comment and detailed explanations. From my experience is the current Bitbucket behavior to delete the merge ref, if the commit is not mergable. So a ls-remote only shows the from and not the merge ref for a PR. The answer why is something not mergable can certainly only be answered by Bitbucket. For example the default merge is valid and Bitbucket denies the merge because of the --ff-only option. So if a build is triggered for the merge and a check or checkout itself reports a missing merge ref, the user have to look at Bitbucket for further information. For the other problem of the lazy update of Bitbucket PR merge ref, this is indeed a problem. I've seen the case, where an update of the from triggers a build via webhook, where the merge ref still points to the old SHA. I don't know how this is solved with other Git repro hosting services like Github, but I think these should be solved at Bitbucket side. So the webhook should trigger a PR build for the from ref, if the from ref is updated and for the merge ref, if the merge ref is updated. Is this possible with the current webhook api?

          Is this possible with the current webhook api?

          AIUI, this is not possible.

          What is probably more productive is to identify the changes to the current merge strategy that would result in a more equivalent merge to that generated by Bitbucket. (right now the only difference I see is the commit message and commit author, which is effectively no difference, but if you can show a case where there is a concrete difference I would view that as a bug that should be fixed)

          Stephen Connolly added a comment - Is this possible with the current webhook api? AIUI, this is not possible. What is probably more productive is to identify the changes to the current merge strategy that would result in a more equivalent merge to that generated by Bitbucket. (right now the only difference I see is the commit message and commit author, which is effectively no difference, but if you can show a case where there is a concrete difference I would view that as a bug that should be fixed)

          Ahhh perhaps you are referring to:

          So what I think is then the case is that we would want to inspect: https://developer.atlassian.com/static/rest/bitbucket-server/5.3.1/bitbucket-rest.html#idm45682777592080

          and parse the 

          "mergeConfig": {
                  "defaultStrategy": {
                      "description": "Always create a merge commit",
                      "enabled": true,
                      "flag": "--no-ff",
                      "id": "no-ff",
                      "name": "Merge commit"
                  },
                  "strategies": [
                      {
                          "description": "Always create a merge commit",
                          "enabled": true,
                          "flag": "--no-ff",
                          "id": "no-ff",
                          "name": "Merge commit"
                      }
                  ],
                  "type": "repository"
              },

          To pick out the "defaultStrategy" and have the merge match that.

           

          It is unclear to me whether the /merge head is the merge from the defaultStrategy or what indeed happens if you enable multiple strategies.

           

          If parsing the "mergeConfig/defaultStrategy" and using that to peform the merge would resolve this issue for you then we should change the JIRA title to reflect the actual requirement, i.e. "Use the repositories defaultStrategy when merging the PR for a PR-merge head" as I do not see us ever checking out the merge head directly given the indications we have received from Atlassian regarding their desires for our implementation

          Stephen Connolly added a comment - Ahhh perhaps you are referring to: So what I think is then the case is that we would want to inspect: https://developer.atlassian.com/static/rest/bitbucket-server/5.3.1/bitbucket-rest.html#idm45682777592080 and parse the  "mergeConfig" : { "defaultStrategy" : { "description" : "Always create a merge commit" , "enabled" : true , "flag" : "--no-ff" , "id" : "no-ff" , "name" : "Merge commit" }, "strategies" : [ { "description" : "Always create a merge commit" , "enabled" : true , "flag" : "--no-ff" , "id" : "no-ff" , "name" : "Merge commit" } ], "type" : "repository" }, To pick out the "defaultStrategy" and have the merge match that.   It is unclear to me whether the /merge head is the merge from the defaultStrategy or what indeed happens if you enable multiple strategies.   If parsing the "mergeConfig/defaultStrategy" and using that to peform the merge would resolve this issue for you then we should change the JIRA title to reflect the actual requirement, i.e. "Use the repositories defaultStrategy when merging the PR for a PR-merge head" as I do not see us ever checking out the merge head directly given the indications we have received from Atlassian regarding their desires for our implementation

          Carsten Kuhl added a comment -

          Yes that was exactly what I was referring to. Bitbucket allows multiple merge strategies with one default for the PRs configured for each repository. The PR-merge head is created locally according to the merge setting.

          But unfortunately during the creation of a PR the user selects one of the allowed strategies, which can be different from the default. And here comes the problem. To rebuild the merge commit on Jenkins side we would need information about the selected PR merge strategy. So the "mergeConfig" of this specific PR and not the default from the general setting.

          I looked deeply into the REST API, but I couldn't find a way to retrieve this kind of information

          So for now I will create an issue on Atlassian side which hopefully shed some light on this issue.

          Carsten Kuhl added a comment - Yes that was exactly what I was referring to. Bitbucket allows multiple merge strategies with one default for the PRs configured for each repository. The PR-merge head is created locally according to the merge setting. But unfortunately during the creation of a PR the user selects one of the allowed strategies, which can be different from the default. And here comes the problem. To rebuild the merge commit on Jenkins side we would need information about the selected PR merge strategy. So the "mergeConfig" of this specific PR and not the default from the general setting. I looked deeply into the REST API, but I couldn't find a way to retrieve this kind of information So for now I will create an issue on Atlassian side which hopefully shed some light on this issue.

          Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Stephen Connolly added a comment - Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          According to https://jira.atlassian.com/browse/BSERV-12284?focusedId=2389584&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2389584, Bitbucket Server 7.0 and subsequent versions no longer create or update "refs/pull-requests/*/merge". Therefore, it would not be useful to make the Bitbucket Branch Source plugin check out those directly, when using Bitbucket Server (previously known as Stash).

          I don't know whether the merge refs are available from Bitbucket Cloud. This issue was filed for Bitbucket Server though, so I'll close this. If Bitbucket Cloud provides the merge refs and someone wants the Bitbucket Branch Source plugin to use them, then I think it'll be best to file a separate issue for that, as the previous comments in this issue do not apply to Bitbucket Cloud.

          Kalle Niemitalo added a comment - According to https://jira.atlassian.com/browse/BSERV-12284?focusedId=2389584&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2389584 , Bitbucket Server 7.0 and subsequent versions no longer create or update "refs/pull-requests/*/merge". Therefore, it would not be useful to make the Bitbucket Branch Source plugin check out those directly, when using Bitbucket Server (previously known as Stash). I don't know whether the merge refs are available from Bitbucket Cloud. This issue was filed for Bitbucket Server though, so I'll close this. If Bitbucket Cloud provides the merge refs and someone wants the Bitbucket Branch Source plugin to use them, then I think it'll be best to file a separate issue for that, as the previous comments in this issue do not apply to Bitbucket Cloud.

          For Bitbucket Cloud, there is a feature request:

          [BCLOUD-5814] [refify-pull-requests-by-making-them-a-ref] Repository refs for pull requests

          It was filed in December 2012 and still doesn't seem to have been implemented.

          Kalle Niemitalo added a comment - For Bitbucket Cloud, there is a feature request: [BCLOUD-5814] [refify-pull-requests-by-making-them-a-ref] Repository refs for pull requests It was filed in December 2012 and still doesn't seem to have been implemented.

          On further thought, the Bitbucket Branch Source plugin was already changed to check out "refs//pull-requests/*/merge", in http://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/174 for JENKINS-48737; this just does not work with Bitbucket Server 7 or higher. I'll mark this issue as fixed, then.

          Kalle Niemitalo added a comment - On further thought, the Bitbucket Branch Source plugin was already changed to check out "refs//pull-requests/*/merge", in http://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/174 for JENKINS-48737 ; this just does not work with Bitbucket Server 7 or higher. I'll mark this issue as fixed, then.

            Unassigned Unassigned
            bbio13 Carsten Kuhl
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: