Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61493

Lightweight checkout fails in Bitbucket Server 7.0; refs/pull-requests/*/merge does not exist

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Labels:
      None
    • Environment:
      Jenkins 2.204.5
      Bitbucket Branch Source Plugin 2.7.0
      Bitbucket Server 7.0.1
    • Similar Issues:

      Description

       

      NOTE

      The linked comment below indicates that in BitBucket 7 Atlassian has removed the ability to do lightweight checkout while using the "Merging" strategy. This is "intentional" and "by-design" behavior implemented by Atlassian. It is beyond the control of Jenkins project to fix this at this time.

      If your build fails while attempting to checkout, be sure you've enabled "Call changes API" for your Bitbucket 7 endpoints.  

      As a workaround to heavyweight checkout, you can try changing to use the "Current Pull Request revision" strategy instead.

      Keep in mind that with "Current Pull Request revision" PR builds it is possible for a PR build to succeed and result in an unstable main branch after changes are merged. If you are an Atlassian customer and do not like this behavior, tell Atlassian this feature is important to you or perhaps someone can make a plugin that can be added to BitBucket server that reintroduces this behavior.

      Suggestions have been made for potential changes on the Jenkins side. These have sounded somewhat complicated or brittle, but PRs are welcome.

      https://jira.atlassian.com/browse/BSERV-12284?focusedCommentId=2389584&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-2389584

       

       

      After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

       ERROR: Could not do lightweight checkout, falling back to heavyweight
       java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
       	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
       	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
       	at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
       	at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
       	at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
       	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
       	at hudson.model.ResourceController.execute(ResourceController.java:97)
       	at hudson.model.Executor.run(Executor.java:427)
      

      git ls-remote origin shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to "the switch from a 3-way diff to a 2-way diff in pull requests" in Bitbucket Server 7.0.0.

      Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

      There might not be any simple way to improve this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree.

        Attachments

          Activity

          Hide
          gmshake Zhenlei Huang added a comment -

          Hi, Liam Newman

          I'd propose reimplement the "lightweight checkout" feature to also work with vendors that does not provide merged refs. A straight simple way is we fetch the target ref and source ref, then merge locally in Jenkins temp workspace, producing the merged version.

          As for single file "lightweight checkout", we can create a temp local git repository, fetch the file from source ref url and target ref url, and then merge them in the created local git repository.

          Compared to the legacy "lightweight checkout", this approach does have cons:

          1. It will need one more http request / git pull request
          2. Need some resources on Jenkins instance to do the merge.

          How do you think this approach?

          Show
          gmshake Zhenlei Huang added a comment - Hi, Liam Newman I'd propose reimplement the "lightweight checkout" feature to also work with vendors that does not provide merged refs. A straight simple way is we fetch the target ref and source ref, then merge locally in Jenkins temp workspace, producing the merged version. As for single file "lightweight checkout", we can create a temp local git repository, fetch the file from source ref url and target ref url, and then merge them in the created local git repository. Compared to the legacy "lightweight checkout", this approach does have cons: It will need one more http request / git pull request Need some resources on Jenkins instance to do the merge. How do you think this approach?
          Hide
          kon Kalle Niemitalo added a comment - - edited

          Zhenlei Huang, I don't think you can merge just two versions of a file such as Jenkinsfile; you need a base version as well. Otherwise, you wouldn't know whether differences mean that lines were added or that they were removed. As I mentioned in the description of this issue, the Bitbucket Server REST API does not seem to provide a way to get the commit ID or file content from the merge base of a pull request. This is why I proposed only implementing the trivial case where the two versions of the file are identical (although the PR has modified other files) and there is no need for an actual merge.

          Show
          kon Kalle Niemitalo added a comment - - edited Zhenlei Huang , I don't think you can merge just two versions of a file such as Jenkinsfile; you need a base version as well. Otherwise, you wouldn't know whether differences mean that lines were added or that they were removed. As I mentioned in the description of this issue, the Bitbucket Server REST API does not seem to provide a way to get the commit ID or file content from the merge base of a pull request. This is why I proposed only implementing the trivial case where the two versions of the file are identical (although the PR has modified other files) and there is no need for an actual merge.
          Hide
          bitwiseman Liam Newman added a comment -

          Kalle Niemitalo
          Thank you for testing that. I can confirm you are correct. Without the "Call Changes API" neither of the merge strategies work at all for new PRs - the `/from` ref is missing and the run fails immediately.

          Zhenlei Huang and Kalle Niemitalo

          A straight simple way is we fetch the target ref and source ref, then merge locally in Jenkins temp workspace, producing the merged version.

          This is why I proposed only implementing the trivial case where the two versions of the file are identical (although the PR has modified other files) and there is no need for an actual merge.

          The solutions you both are advocating are both possible, but I would not advise them. The number of edge cases (and not-so-edge complexities) in this area is high. Any solution that doesn't use an scm-provider-side solution, is likely be a bug farm.
          Zhenlei Huang - Kalle pointed to some limitation to what you proposed. Also, what about forks? The current implementation expects to find all files in the target repo, switching to a fork for some would be problematic.
          Kalle Niemitalo Your proposal is simpler but it would only work if the pipeline does not call any of the steps that do additional reads (readTrusted, etc), and there is no way to know if those steps are present until during the run.

          Overall, what you are suggesting is doing a lot of work to mitigate something that is almost trivial to do on on the SCM server. You would be better served by creating a plugin/app for Bitbucket that implements the merge ref creation.

          If you would like to implement and provide indefinite support for this case, I'm open to taking PRs that add functionality as long is it off by default.

          Show
          bitwiseman Liam Newman added a comment - Kalle Niemitalo Thank you for testing that. I can confirm you are correct. Without the "Call Changes API" neither of the merge strategies work at all for new PRs - the `/from` ref is missing and the run fails immediately. Zhenlei Huang and Kalle Niemitalo A straight simple way is we fetch the target ref and source ref, then merge locally in Jenkins temp workspace, producing the merged version. This is why I proposed only implementing the trivial case where the two versions of the file are identical (although the PR has modified other files) and there is no need for an actual merge. The solutions you both are advocating are both possible, but I would not advise them. The number of edge cases (and not-so-edge complexities) in this area is high. Any solution that doesn't use an scm-provider-side solution, is likely be a bug farm. Zhenlei Huang - Kalle pointed to some limitation to what you proposed. Also, what about forks? The current implementation expects to find all files in the target repo, switching to a fork for some would be problematic. Kalle Niemitalo Your proposal is simpler but it would only work if the pipeline does not call any of the steps that do additional reads (readTrusted, etc), and there is no way to know if those steps are present until during the run. Overall, what you are suggesting is doing a lot of work to mitigate something that is almost trivial to do on on the SCM server. You would be better served by creating a plugin/app for Bitbucket that implements the merge ref creation. If you would like to implement and provide indefinite support for this case, I'm open to taking PRs that add functionality as long is it off by default.
          Hide
          kon Kalle Niemitalo added a comment -

          Liam Newman, I guess I should then file a separate request to allow disabling Git LFS in the "Merging the pull request with the current target branch revision" heavyweight checkout at the Jenkins master. That should at least be simple to implement and would reduce the disk space and network I/O costs in my scenario.

          Show
          kon Kalle Niemitalo added a comment - Liam Newman , I guess I should then file a separate request to allow disabling Git LFS in the "Merging the pull request with the current target branch revision" heavyweight checkout at the Jenkins master. That should at least be simple to implement and would reduce the disk space and network I/O costs in my scenario.
          Hide
          gmshake Zhenlei Huang added a comment -

          There's another edge case not covered by my previous proposal. Say we have a file named 'Jenkinsfile.old' in base branch, and it was renamed to 'Jenkinsfile' in PR's head branch, it looks difficult to get the original file name unless SCM server provide the information in the PR. 

          Liam Newman,
          I agree with you that it is trivial to do on the SCM server. It's frustrating to hack on Jenkins' side

          Show
          gmshake Zhenlei Huang added a comment - There's another edge case not covered by my previous proposal. Say we have a file named 'Jenkinsfile.old' in base branch, and it was renamed to 'Jenkinsfile' in PR's head branch, it looks difficult to get the original file name unless SCM server provide the information in the PR.  Liam Newman , I agree with you that it is trivial to do on the SCM server. It's frustrating to hack on Jenkins' side

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            kon Kalle Niemitalo
            Votes:
            11 Vote for this issue
            Watchers:
            15 Start watching this issue

              Dates

              Created:
              Updated: