Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38973

Pipeline initial checkout of Jenkinsfile not honoring git timeout

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: git-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.19.1
      Git client plugin 2.0.0
      Git plugin 3.0.0
      Pipeline: Multibranch: 2.9
    • Similar Issues:

      Description

      I am having an issue cloning a large repo for use in a multi-branch pipeline project.

      My goal is to run pipeline jobs against certain branches within my repository. However, the initial fetch seems to timeout. As you can see from the attached log, the issue appears to arise from a 10 minute timeout issue. However, if you see the below screen shots, I have all of my timeouts set to 180minutes.

      In the attached tickets there were known issues with the 10 minute git timeout that were fixed by allowing users access to overrides as I have shown in the attached images. It appears as though these overrides are not being honored for some reason within the MultibranchPipeline project.

      I have used the settings described here on a smaller repo and it was able to properly index the branches.

      As a work around, I also tried only using the overrides as advanced settings for checkout/fetch behavior. This did not work either, unfortunately.

        Attachments

          Issue Links

            Activity

            Hide
            rohit_gagan Rohit Gagan added a comment -

            Mark Waite. Tried adding that, this still takes timeout as 10 min

             

            Started
            [Wed May 22 11:55:15 EDT 2019] Starting branch indexing...
            > git --version # timeout=10

            Show
            rohit_gagan Rohit Gagan added a comment - Mark Waite . Tried adding that, this still takes timeout as 10 min   Started [Wed May 22 11:55:15 EDT 2019] Starting branch indexing... > git --version # timeout=10
            Hide
            rohit_gagan Rohit Gagan added a comment -

            Mark Waite only option i see is change in git plugin code then to increase the default timeout. What you suggest?

             

            Show
            rohit_gagan Rohit Gagan added a comment - Mark Waite only option i see is change in git plugin code then to increase the default timeout. What you suggest?  
            Hide
            rohit_gagan Rohit Gagan added a comment -

            Mark Waite. Thanks. This works. i was adding the parameter at the wrong end

            Show
            rohit_gagan Rohit Gagan added a comment - Mark Waite . Thanks. This works. i was adding the parameter at the wrong end
            Hide
            glatapoui Joel added a comment -

            I have this issue too (index-pack died of signal 15) because our repo is too big (20+GB), even though I gave lightweight checkout enabled.

            I'm wondering why Jenkins is not using git archive as an alternative to checking out the whole repository.

            The current method could still be kept as a fallback if the remote has not enabled that option (like for GitHub, for example)

            Show
            glatapoui Joel added a comment - I have this issue too ( index-pack died of signal 15 ) because our repo is too big (20+GB), even though I gave lightweight checkout enabled. I'm wondering why Jenkins is not using git archive  as an alternative to checking out the whole repository. The current method could still be kept as a fallback if the remote has not enabled that option (like for GitHub , for example)
            Hide
            markewaite Mark Waite added a comment -

            Joel refer to "Git in the Large - Tips and Tricks" for recommendations of techniques you can use to reduce the challenges of large git repositories. The techniques that have been most effective for me have included (in order of effectiveness):

            1. Use git large file support rather than storing large files directly in git
            2. Place a reference copy of the repository on the agent that is receiving the large repository and update it periodically (weekly, monthly)
            3. Narrow the refspec used in the job definition so that only the necessary branches are fetched from the remote to the local repository (can be a very significant win if there are many branches in the repository and the content of the branches is mostly distinct from other branches)
            4. Use shallow clone to limit the history depth that is retrieved (sometimes this is known to place too heavy a load on the remote server and should not be used)
            5. Use sparse checkout to reduce the disc use for the agent checkout

            I don't plan to implement git archive support in the git plugin.

            Show
            markewaite Mark Waite added a comment - Joel refer to "Git in the Large - Tips and Tricks" for recommendations of techniques you can use to reduce the challenges of large git repositories. The techniques that have been most effective for me have included (in order of effectiveness): Use git large file support rather than storing large files directly in git Place a reference copy of the repository on the agent that is receiving the large repository and update it periodically (weekly, monthly) Narrow the refspec used in the job definition so that only the necessary branches are fetched from the remote to the local repository (can be a very significant win if there are many branches in the repository and the content of the branches is mostly distinct from other branches) Use shallow clone to limit the history depth that is retrieved (sometimes this is known to place too heavy a load on the remote server and should not be used) Use sparse checkout to reduce the disc use for the agent checkout I don't plan to implement git archive support in the git plugin.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              magic David Johnson
              Votes:
              24 Vote for this issue
              Watchers:
              32 Start watching this issue

                Dates

                Created:
                Updated: