Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61493

Lightweight checkout fails in Bitbucket Server 7.0; refs/pull-requests/*/merge does not exist

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • Jenkins 2.204.5
      Bitbucket Branch Source Plugin 2.7.0
      Bitbucket Server 7.0.1

       

      NOTE

      The linked comment below indicates that in BitBucket 7 Atlassian has removed the ability to do lightweight checkout while using the "Merging" strategy. This is "intentional" and "by-design" behavior implemented by Atlassian. It is beyond the control of Jenkins project to fix this at this time.

      If your build fails while attempting to checkout, be sure you've enabled "Call changes API" for your Bitbucket 7 endpoints.  

      As a workaround to heavyweight checkout, you can try changing to use the "Current Pull Request revision" strategy instead.

      Keep in mind that with "Current Pull Request revision" PR builds it is possible for a PR build to succeed and result in an unstable main branch after changes are merged. If you are an Atlassian customer and do not like this behavior, tell Atlassian this feature is important to you or perhaps someone can make a plugin that can be added to BitBucket server that reintroduces this behavior.

      Suggestions have been made for potential changes on the Jenkins side. These have sounded somewhat complicated or brittle, but PRs are welcome.

      https://jira.atlassian.com/browse/BSERV-12284?focusedCommentId=2389584&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-2389584

       

       

      After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

       ERROR: Could not do lightweight checkout, falling back to heavyweight
       java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
       	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
       	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
       	at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
       	at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
       	at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
       	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
       	at hudson.model.ResourceController.execute(ResourceController.java:97)
       	at hudson.model.Executor.run(Executor.java:427)
      

      git ls-remote origin shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to "the switch from a 3-way diff to a 2-way diff in pull requests" in Bitbucket Server 7.0.0.

      Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

      There might not be any simple way to improve this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree.

          [JENKINS-61493] Lightweight checkout fails in Bitbucket Server 7.0; refs/pull-requests/*/merge does not exist

          Kalle Niemitalo created issue -
          Kalle Niemitalo made changes -
          Summary Original: Lightweight checkout fails in Bitbucket Server 7.0; refs/pull-requests/*/from does not exist New: Lightweight checkout fails in Bitbucket Server 7.0; refs/pull-requests/*/merge does not exist
          Kalle Niemitalo made changes -
          Description Original: After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

          {noformat}
           ERROR: Could not do lightweight checkout, falling back to heavyweight
           java.io.FileNotFoundException: URL: /rest/api/1.0/projects/SISU/repos/rahti-hybrid/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
            at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
            at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
            at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
            at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:427)
          {noformat}

          {{git ls-remote origin}} shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to ["the switch from a 3-way diff to a 2-way diff in pull requests"|https://confluence.atlassian.com/bitbucketserver/bitbucket-server-7-0-release-notes-990546638.html#BitbucketServer7.0releasenotes-Amodifiedbehaviorfordiffs] in Bitbucket Server 7.0.0.

          Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

          I don't know whether there is any simple way to fix this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree. That would require a REST API for getting the commit ID of a merge base though, and I don't know whether such a thing exists.
          New: After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

          {noformat}
           ERROR: Could not do lightweight checkout, falling back to heavyweight
           java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
            at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
            at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
            at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
            at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:427)
          {noformat}

          {{git ls-remote origin}} shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to ["the switch from a 3-way diff to a 2-way diff in pull requests"|https://confluence.atlassian.com/bitbucketserver/bitbucket-server-7-0-release-notes-990546638.html#BitbucketServer7.0releasenotes-Amodifiedbehaviorfordiffs] in Bitbucket Server 7.0.0.

          Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

          I don't know whether there is any simple way to fix this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree. That would require a REST API for getting the commit ID of a merge base though, and I don't know whether such a thing exists.
          Kalle Niemitalo made changes -
          Description Original: After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

          {noformat}
           ERROR: Could not do lightweight checkout, falling back to heavyweight
           java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
            at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
            at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
            at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
            at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:427)
          {noformat}

          {{git ls-remote origin}} shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to ["the switch from a 3-way diff to a 2-way diff in pull requests"|https://confluence.atlassian.com/bitbucketserver/bitbucket-server-7-0-release-notes-990546638.html#BitbucketServer7.0releasenotes-Amodifiedbehaviorfordiffs] in Bitbucket Server 7.0.0.

          Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

          I don't know whether there is any simple way to fix this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree. That would require a REST API for getting the commit ID of a merge base though, and I don't know whether such a thing exists.
          New: After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

          {noformat}
           ERROR: Could not do lightweight checkout, falling back to heavyweight
           java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
            at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
            at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
            at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
            at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:427)
          {noformat}

          {{git ls-remote origin}} shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to ["the switch from a 3-way diff to a 2-way diff in pull requests"|https://confluence.atlassian.com/bitbucketserver/bitbucket-server-7-0-release-notes-990546638.html#BitbucketServer7.0releasenotes-Amodifiedbehaviorfordiffs] in Bitbucket Server 7.0.0.

          Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

          There might not be any simple way to improve this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree.

           - One way might be to use the REST API to get Jenkinsfile from the target branch ([/rest/api/1.0/projects/\{projectKey\}/repos/\{repositorySlug\}/browse/\{path:.*\}|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp208]) and any Jenkinsfile changes of the pull request as a patch ([/rest/api/1.0/projects/\{projectKey\}/repos/\{repositorySlug\}/pull-requests/\{pullRequestId\}.patch|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp289] gives the standard diff format but does not support limiting to a single file; [/rest/api/1.0/projects/\{projectKey}/repos/\{repositorySlug}/pull-requests/\{pullRequestId}/diff/\{path:.*}|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp229] supports that but the output is JSON) and then apply the patch to Jenkinsfile at the Jenkins master.
           - Alternatively, if Bitbucket Server has a REST API for querying the commit ID of a merge base of a pull request, then the plugin could first use that, then get Jenkinsfile from the three commits, and finally do the three-way merge. However, there does not seem to be such an API.

          Mentioned in BSERV-11477 (comment).

          Kalle Niemitalo added a comment - Mentioned in BSERV-11477 (comment) .
          Kalle Niemitalo made changes -
          Environment New: Jenkins 2.204.5
          Bitbucket Branch Source Plugin 2.7.0
          Bitbucket Server 7.0.1
          Kalle Niemitalo made changes -
          Description Original: After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

          {noformat}
           ERROR: Could not do lightweight checkout, falling back to heavyweight
           java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
            at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
            at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
            at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
            at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:427)
          {noformat}

          {{git ls-remote origin}} shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to ["the switch from a 3-way diff to a 2-way diff in pull requests"|https://confluence.atlassian.com/bitbucketserver/bitbucket-server-7-0-release-notes-990546638.html#BitbucketServer7.0releasenotes-Amodifiedbehaviorfordiffs] in Bitbucket Server 7.0.0.

          Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

          There might not be any simple way to improve this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree.

           - One way might be to use the REST API to get Jenkinsfile from the target branch ([/rest/api/1.0/projects/\{projectKey\}/repos/\{repositorySlug\}/browse/\{path:.*\}|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp208]) and any Jenkinsfile changes of the pull request as a patch ([/rest/api/1.0/projects/\{projectKey\}/repos/\{repositorySlug\}/pull-requests/\{pullRequestId\}.patch|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp289] gives the standard diff format but does not support limiting to a single file; [/rest/api/1.0/projects/\{projectKey}/repos/\{repositorySlug}/pull-requests/\{pullRequestId}/diff/\{path:.*}|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp229] supports that but the output is JSON) and then apply the patch to Jenkinsfile at the Jenkins master.
           - Alternatively, if Bitbucket Server has a REST API for querying the commit ID of a merge base of a pull request, then the plugin could first use that, then get Jenkinsfile from the three commits, and finally do the three-way merge. However, there does not seem to be such an API.
          New: After upgrading to Bitbucket Server 7.0.1, we get this kind of error for every pull request in multibranch projects in which Jenkins has been configured to build the result of merging the pull request:

          {noformat}
           ERROR: Could not do lightweight checkout, falling back to heavyweight
           java.io.FileNotFoundException: URL: /rest/api/1.0/projects/REDACTED/repos/redacted/browse/Jenkinsfile?at=pull-requests%2F771%2Fmerge&start=0&limit=500
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:831)
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getFileContent(BitbucketServerAPIClient.java:1123)
            at com.cloudbees.jenkins.plugins.bitbucket.filesystem.BitbucketSCMFile.content(BitbucketSCMFile.java:98)
            at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
            at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
            at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:427)
          {noformat}

          {{git ls-remote origin}} shows that Bitbucket Server now publishes "refs/pull-requests/771/from" but not "refs/pull-requests/771/merge", even though the pull request has no merge conflicts (the target branch even has no new commits). This might be related to ["the switch from a 3-way diff to a 2-way diff in pull requests"|https://confluence.atlassian.com/bitbucketserver/bitbucket-server-7-0-release-notes-990546638.html#BitbucketServer7.0releasenotes-Amodifiedbehaviorfordiffs] in Bitbucket Server 7.0.0.

          Jenkins then does the heavyweight checkout and successfully merges the pull request, so the project ultimately builds OK, but the heavyweight checkout consumes a lot of time and disk space.

          There might not be any simple way to improve this in the bitbucket-branch-source plugin. If Bitbucket Server has deliberately ceased publishing "refs/pull-requests/*/merge", perhaps the plugin could do a three-way merge on Jenkinsfile only, rather than on the entire tree.

           - One way might be to use the REST API to get Jenkinsfile from the target branch ([/rest/api/1.0/projects/\{projectKey\}/repos/\{repositorySlug\}/browse/\{path:.*\}|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp208]) and any Jenkinsfile changes of the pull request as a patch, and then apply the patch to Jenkinsfile at the Jenkins master. Unfortunately, it does not seem easy to get a patch that can be applied:
           ** [/rest/api/1.0/projects/\{projectKey\}/repos/\{repositorySlug\}/pull-requests/\{pullRequestId\}.patch|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp289] gives the standard diff format but does not support limiting to a single file.
           ** [/rest/api/1.0/projects/\{projectKey}/repos/\{repositorySlug}/pull-requests/\{pullRequestId}/diff/\{path:.*}|https://docs.atlassian.com/bitbucket-server/rest/7.0.1/bitbucket-rest.html#idp229] supports limiting to a single file, but the output is JSON and would thus be more difficult to apply.
           - Alternatively, if Bitbucket Server has a REST API for querying the commit ID of a merge base of a pull request, then the plugin could first use that, then get Jenkinsfile from the three commits, and finally do the three-way merge. However, there does not seem to be such an API.

          Alexey Pelykh added a comment -

          Seems that on 7.1.0 "/from" are not revealed via ls-remove as well

          Alexey Pelykh added a comment - Seems that on 7.1.0 "/from" are not revealed via ls-remove as well

          Alexey Pelykh added a comment -

          It seems that unless non-author user actually tries to view the PR, the "/from" ref is not created. So when I just use PRs on my own - "/from" is not created whatsoever, that's seems to be a severe case

          Alexey Pelykh added a comment - It seems that unless non-author user actually tries to view the PR, the "/from" ref is not created. So when I just use PRs on my own - "/from" is not created whatsoever, that's seems to be a severe case

          Alexey Pelykh added a comment -

          Alexey Pelykh added a comment - Seems  https://jira.atlassian.com/browse/BSERV-12198?focusedCommentId=2374169  is relatede as well

            Unassigned Unassigned
            kon Kalle Niemitalo
            Votes:
            15 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated: