Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62349

Git plugin lightweight checkout copies full repository

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Using multibranch pipeline clones the entire repository on the master instead of just getting the Jenkinsfile with the lighweight method. The lightweight checkout is the default behaviour as far as I know, but it's not working in my case. The clone is repeated when the pipeline starts on a slave machine, of course. Here is my setup:

      Branch Sources / Bitbucket server (connection is fine)

      Behaviours:

      Discover branches

      Advanced clone behaviours:

      • Fetch tags: unchecked
      • Honor refspec on initial clone: checked
      • Shallow clone: checked
      • Shallow clone depth: 1
      • Path to reference repo to use during clone: empty
      • Timeout: empty

      Build strategies: Skip initial build on first branch indexing

      Build configuration:

      • Mode: by Jenkinsfile
      • Script path: build/jenkins/compile/Jenkinsfile

      Please find the checkout log in the attachment.

        Attachments

          Issue Links

            Activity

            gpilisi Gergely Pilisi created issue -
            markewaite Mark Waite made changes -
            Field Original Value New Value
            Summary Multibranch pipeline lightweight checkout not working Git plugin lightweight checkout copies full repository
            Hide
            markewaite Mark Waite added a comment - - edited

            Command line git does not provide a way to request a single file from a remote repository. The mode of operation is to fetch the remote repository locally and then checkout the needed file locally.

            Lightweight checkout is most useful with higher level git providers like GitHub,, Bitbucket, GitLab, and Gitea. Those higher level providers use a different access technique to request the Jenkinsfile. The git plugin could use the GitRepositoryBrowser information which is optionally provided by the user to attempt to request a single file from a specific branch or revision. That would require writing the code to formulate the request for a single file to each of the repository browser providers and would require that the user must provide that repository browser URL in the job definition. That has not been done.

            This is a good enhancement request, though it is not high priority on my list of enhancements.

            Since you're using Bitbucket, I believe you could get the benefits of the Bitbucket API by using a multibranch pipeline for that repository, then define it to monitor only a single branch.

            Show
            markewaite Mark Waite added a comment - - edited Command line git does not provide a way to request a single file from a remote repository. The mode of operation is to fetch the remote repository locally and then checkout the needed file locally. Lightweight checkout is most useful with higher level git providers like GitHub,, Bitbucket, GitLab, and Gitea. Those higher level providers use a different access technique to request the Jenkinsfile. The git plugin could use the GitRepositoryBrowser information which is optionally provided by the user to attempt to request a single file from a specific branch or revision. That would require writing the code to formulate the request for a single file to each of the repository browser providers and would require that the user must provide that repository browser URL in the job definition. That has not been done. This is a good enhancement request, though it is not high priority on my list of enhancements. Since you're using Bitbucket, I believe you could get the benefits of the Bitbucket API by using a multibranch pipeline for that repository, then define it to monitor only a single branch.
            markewaite Mark Waite made changes -
            Assignee Mark Waite [ markewaite ]
            markewaite Mark Waite made changes -
            Description Using multibranch pipeline clones the entire repository on the master instead of just getting the Jenkinsfile with the lighweight method. The lightweight checkout is the default behaviour as far as I know, but it's not working in my case. The clone is repeated when the pipeline starts on a slave machine, of course. Here is my setup:

            Branch Sources / Bitbucket server (connection is fine)

            Behaviours:

            Discover branches

            Advanced clone behaviours:

            -Fetch tags: unchecked

            -Honor refspec on initial clone: checked

            -Shallow clone: checked

            -Shallow clone depth: 1

            -Path to reference repo to use during clone: empty

            -Timeout: empty

            Build strategies: Skip initial build on first branch indexing

            Build configuration:

            -Mode: by Jenkinsfile

            -Script path: build/jenkins/compile/Jenkinsfile

            Please find the checkout log in the attachment.

             
            Using multibranch pipeline clones the entire repository on the master instead of just getting the Jenkinsfile with the lighweight method. The lightweight checkout is the default behaviour as far as I know, but it's not working in my case. The clone is repeated when the pipeline starts on a slave machine, of course. Here is my setup:

            Branch Sources / Bitbucket server (connection is fine)

            Behaviours:

            Discover branches

            Advanced clone behaviours:

            - Fetch tags: unchecked
            - Honor refspec on initial clone: checked
            - Shallow clone: checked
            - Shallow clone depth: 1
            - Path to reference repo to use during clone: empty
            - Timeout: empty

            Build strategies: Skip initial build on first branch indexing

            Build configuration:

            - Mode: by Jenkinsfile
            - Script path: build/jenkins/compile/Jenkinsfile

            Please find the checkout log in the attachment.
            Hide
            gpilisi Gergely Pilisi added a comment -

            Thanks for the info. However I can see a couple of tickets, like this:

            Provide lightweight checkout capability for bitbucket to avoid repository clone for multi-branch pipeline jobs

            It's in RESOLVED state. I thought that this feature is ready nowadays.

             

            Show
            gpilisi Gergely Pilisi added a comment - Thanks for the info. However I can see a couple of tickets, like this: Provide lightweight checkout capability for bitbucket to avoid repository clone for multi-branch pipeline jobs It's in RESOLVED state. I thought that this feature is ready nowadays.  
            Hide
            markewaite Mark Waite added a comment -

            Gergely Pilisi it is resolved and works great. It is for multibranch pipelines. The job definition you are describing seems (to me) to not be a multibranch pipeline. It is a pipeline job that is not a multibranch pipeline.

            If you're using a multibranch pipeline, then you need to use the Bitbucket provider in that multibranch pipeline rather than using the git provider. The Bitbucket provider is a higher level implementation than the git plugin. It knows it is communicating with Bitbucket and can use the Bitbucket REST API to optimize certain operations (like lightweight checkout and change detection).

            Show
            markewaite Mark Waite added a comment - Gergely Pilisi it is resolved and works great. It is for multibranch pipelines . The job definition you are describing seems (to me) to not be a multibranch pipeline. It is a pipeline job that is not a multibranch pipeline. If you're using a multibranch pipeline, then you need to use the Bitbucket provider in that multibranch pipeline rather than using the git provider. The Bitbucket provider is a higher level implementation than the git plugin. It knows it is communicating with Bitbucket and can use the Bitbucket REST API to optimize certain operations (like lightweight checkout and change detection).
            Hide
            gpilisi Gergely Pilisi added a comment -

            It is a multibranch pipeline. And I use Bitbucket Server as a branch source. These are the opitons, I think it is a valid Bitbucket provider, not a simple git repo:

            Credentials (for build auth): (our tehcnical user's credential)
            Bitbucket Server instance: (the name of our Bitbucket instance which is configured under Configure System / Bitbucket Server Integration)
            Project name: (name of the project which contains our repo)
            Repository name: (the name of the repo whihc contains the Jenkinsfile too)
            Clone from: Primary Server
            Behaviours: as mentioned above
            etc.

            Show
            gpilisi Gergely Pilisi added a comment - It is a multibranch pipeline. And I use Bitbucket Server as a branch source. These are the opitons, I think it is a valid Bitbucket provider, not a simple git repo: Credentials (for build auth): (our tehcnical user's credential) Bitbucket Server instance: (the name of our Bitbucket instance which is configured under Configure System / Bitbucket Server Integration) Project name: (name of the project which contains our repo) Repository name: (the name of the repo whihc contains the Jenkinsfile too) Clone from: Primary Server Behaviours: as mentioned above etc.
            Hide
            gpilisi Gergely Pilisi added a comment -

            Maybe I choosed a wrong component, trying the bitbucket-branch-source-plugin now.

            Show
            gpilisi Gergely Pilisi added a comment - Maybe I choosed a wrong component, trying the bitbucket-branch-source-plugin now.
            gpilisi Gergely Pilisi made changes -
            Component/s bitbucket-branch-source-plugin [ 21428 ]
            Component/s git-plugin [ 15543 ]
            Component/s pipeline [ 21692 ]
            Labels git pipeline plugin bitbucket bitbucket-branch-source-plugin bitbucket-server
            Hide
            carlosrodlop Carlos Rodríguez López added a comment -

            Gergely Pilisi see https://issues.jenkins.io/browse/JENKINS-61493 .

            This ticket seems a duplication

            Show
            carlosrodlop Carlos Rodríguez López added a comment - Gergely Pilisi see https://issues.jenkins.io/browse/JENKINS-61493 . This ticket seems a duplication
            Hide
            kon Kalle Niemitalo added a comment - - edited

            JENKINS-61493 is for lightweight checkout in the Bitbucket Branch Source plugin when building the result of merging a pull request. I don't see any git merge command in checkout_log, so this is a different issue.

            If the branch source is "Bitbucket server" rather than "Bitbucket", then you are using the Bitbucket Server Integration plugin rather than Bitbucket Branch Source, and this issue is a duplicate of JENKINS-63033, which is currently FIXED BUT UNRELEASED.

            Show
            kon Kalle Niemitalo added a comment - - edited JENKINS-61493 is for lightweight checkout in the Bitbucket Branch Source plugin when building the result of merging a pull request . I don't see any git merge command in checkout_log , so this is a different issue. If the branch source is "Bitbucket server" rather than "Bitbucket", then you are using the Bitbucket Server Integration plugin rather than Bitbucket Branch Source, and this issue is a duplicate of JENKINS-63033 , which is currently FIXED BUT UNRELEASED.
            Hide
            kon Kalle Niemitalo added a comment -

            Credentials (for build auth): (our tehcnical user's credential)

            That setting exists in "Bitbucket server" branch sources (Bitbucket Server Integration plugin), not in "Bitbucket" branch sources (Bitbucket Branch Source plugin). Therefore, I'll go ahead and move this to the atlassian-bitbucket-server-integration-plugin component and resolve as a duplicate of JENKINS-63033.

            Show
            kon Kalle Niemitalo added a comment - Credentials (for build auth): (our tehcnical user's credential) That setting exists in "Bitbucket server" branch sources (Bitbucket Server Integration plugin), not in "Bitbucket" branch sources (Bitbucket Branch Source plugin). Therefore, I'll go ahead and move this to the atlassian-bitbucket-server-integration-plugin component and resolve as a duplicate of JENKINS-63033 .
            kon Kalle Niemitalo made changes -
            Component/s atlassian-bitbucket-server-integration-plugin [ 25721 ]
            Component/s bitbucket-branch-source-plugin [ 21428 ]
            kon Kalle Niemitalo made changes -
            Link This issue duplicates JENKINS-63033 [ JENKINS-63033 ]
            kon Kalle Niemitalo made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Fixed but Unreleased [ 10203 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              gpilisi Gergely Pilisi
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: