Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63033

Implement Lightweight Checkout for Multibranch Pipeline Jobs

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      3.0.0

      Description

      I tried to migrate a multibranch project from Bitbucket Branch Source Plugin to Bitbucket Server Integration. Before the migration, the Jenkins master read just the Jenkinsfile of each branch from Bitbucket Server over HTTP, and the Jenkins agent then cloned the repository and submodules and pulled Git LFS files. After the migration, the Jenkins master cloned all branches of the repository and pulled Git LFS files. We cannot afford to let the Jenkins master do that, because of time and disk space. I had to revert the migration for this project.

      Please change the Bitbucket Server Integration plugin to support lightweight checkout for Jenkinsfile. Perhaps that is part of "SCM API 2.0" JENKINS-62002; I don't know.

        Attachments

          Issue Links

            Activity

            Hide
            kon Kalle Niemitalo added a comment -

            Now that JENKINS-63069 has been fixed, this JENKINS-63033 is the only known issue blocking our migration from Bitbucket Branch Source Plugin.

            Show
            kon Kalle Niemitalo added a comment - Now that JENKINS-63069 has been fixed, this JENKINS-63033 is the only known issue blocking our migration from Bitbucket Branch Source Plugin.
            Hide
            kon Kalle Niemitalo added a comment -

            Lightweight checkout in Bitbucket Branch Source Plugin was requested in JENKINS-42518 and implemented in PR#78. The diffstat was:

            58      0       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/Utils.java
            23      1       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/api/BitbucketApi.java
            57      5       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/client/BitbucketCloudApiClient.java
            82      0       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/client/repository/BitbucketRepositorySource.java
            104     0       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/filesystem/BitbucketSCMFile.java
            147     0       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/filesystem/BitbucketSCMFileSystem.java
            81      10      src/main/java/com/cloudbees/jenkins/plugins/bitbucket/server/client/BitbucketServerAPIClient.java
            4       0       src/main/java/com/cloudbees/jenkins/plugins/bitbucket/server/client/BitbucketServerApiFactory.java
            

            The main thing seems to be class BitbucketSCMFileSystem (source, Javadoc), which extends SCMFileSystem (source, Javadoc). In contrast, the source code of Bitbucket Server Integration never mentions SCMFileSystem.

            From SCM API Implementation Guide, I get the impression that SCMProbe (source, Javadoc) is an alternative to SCMFileSystem. Because the source code of Bitbucket Branch Source Plugin does not mention SCMProbe, I think Bitbucket Server Integration need not implement that.

            The BitbucketServerAPIClient.getFileContent method in Bitbucket Branch Source Plugin uses the "/rest/api/1.0/projects/{owner}/repos/{repo}/browse{/path*}{?at}" (REST API doc) URI template and apparently supports UTF-8 text files only. I wonder why that was chosen instead of "/rest/api/1.0/projects/{owner}/repos/{repo}/raw{/path*}{?at}" (REST API doc), which would respond with an octet stream.

            Show
            kon Kalle Niemitalo added a comment - Lightweight checkout in Bitbucket Branch Source Plugin was requested in JENKINS-42518 and implemented in PR#78 . The diffstat was: 58 0 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/Utils.java 23 1 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/api/BitbucketApi.java 57 5 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/client/BitbucketCloudApiClient.java 82 0 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/client/repository/BitbucketRepositorySource.java 104 0 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/filesystem/BitbucketSCMFile.java 147 0 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/filesystem/BitbucketSCMFileSystem.java 81 10 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/server/client/BitbucketServerAPIClient.java 4 0 src/main/java/com/cloudbees/jenkins/plugins/bitbucket/server/client/BitbucketServerApiFactory.java The main thing seems to be class BitbucketSCMFileSystem ( source , Javadoc ), which extends SCMFileSystem ( source , Javadoc ). In contrast, the source code of Bitbucket Server Integration never mentions SCMFileSystem . From SCM API Implementation Guide , I get the impression that SCMProbe ( source , Javadoc ) is an alternative to SCMFileSystem . Because the source code of Bitbucket Branch Source Plugin does not mention SCMProbe , I think Bitbucket Server Integration need not implement that. The BitbucketServerAPIClient.getFileContent method in Bitbucket Branch Source Plugin uses the "/rest/api/1.0/projects/{owner}/repos/{repo}/browse{/path*}{?at}" ( REST API doc ) URI template and apparently supports UTF-8 text files only. I wonder why that was chosen instead of "/rest/api/1.0/projects/{owner}/repos/{repo}/raw{/path*}{?at}" ( REST API doc ), which would respond with an octet stream.
            Hide
            macdrega Joerg Schwaerzler added a comment - - edited

            Will this ticket cover lightweight checkout for pull requests, too? See also: JENKINS-60342

            Show
            macdrega Joerg Schwaerzler added a comment - - edited Will this ticket cover lightweight checkout for pull requests, too? See also: JENKINS-60342
            Hide
            kon Kalle Niemitalo added a comment -

            This would just bring the same capability that the Bitbucket Branch Source plugin already has. Lightweight checkout for the result of merging a pull request no longer works in the Bitbucket Branch Source plugin used with Bitbucket Server (JENKINS-61493), and the Bitbucket Server Integration plugin would then have the same restriction.

            Show
            kon Kalle Niemitalo added a comment - This would just bring the same capability that the Bitbucket Branch Source plugin already has. Lightweight checkout for the result of merging a pull request no longer works in the Bitbucket Branch Source plugin used with Bitbucket Server ( JENKINS-61493 ), and the Bitbucket Server Integration plugin would then have the same restriction.
            Hide
            mhenschke_atlassian Martin Henschke added a comment - - edited

            I am spiking this ticket at the moment. For anyone watching/waiting on this ticket, are you interested in this feature for pipeline jobs (as opposed to Multibranch Pipeline jobs)? I have been spiking the solution and am not confident a functionally complete solution is possible given how pipeline jobs are implemented, so we're considering just releasing support for Multibranch for now. If anyone is interested in this for pipeline specifically, please file a separate ticket.
            EDIT: After a bit more investigation I've confirmed it's impossible for every case, but we can do pipeline so long as the branch selector points to a single branch (no wildcards/selectors)- it appears this is what the GitSCM implementation does so we'll follow suit.

            Show
            mhenschke_atlassian Martin Henschke added a comment - - edited I am spiking this ticket at the moment. For anyone watching/waiting on this ticket, are you interested in this feature for pipeline jobs (as opposed to Multibranch Pipeline jobs)? I have been spiking the solution and am not confident a functionally complete solution is possible given how pipeline jobs are implemented, so we're considering just releasing support for Multibranch for now. If anyone is interested in this for pipeline specifically, please file a separate ticket. EDIT: After a bit more investigation I've confirmed it's impossible for every case, but we can do pipeline so long as the branch selector points to a single branch (no wildcards/selectors)- it appears this is what the GitSCM implementation does so we'll follow suit.
            Hide
            kon Kalle Niemitalo added a comment - - edited
            Show
            kon Kalle Niemitalo added a comment - - edited Martin Henschke , what do you mean with "spiking"? I imagine it is not "5. (figuratively, journalism) To discard; to decide not to publish or make public."
            Hide
            mhenschke_atlassian Martin Henschke added a comment -

            Kalle Niemitalo quite the opposite! I meant spike in the sense of minimal prototyping.

            Show
            mhenschke_atlassian Martin Henschke added a comment - Kalle Niemitalo quite the opposite! I meant spike in the sense of minimal prototyping .

              People

              Assignee:
              mhenschke_atlassian Martin Henschke
              Reporter:
              kon Kalle Niemitalo
              Votes:
              7 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: