Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38973

Pipeline initial checkout of Jenkinsfile not honoring git timeout

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • git-plugin
    • None
    • Jenkins 2.19.1
      Git client plugin 2.0.0
      Git plugin 3.0.0
      Pipeline: Multibranch: 2.9

      I am having an issue cloning a large repo for use in a multi-branch pipeline project.

      My goal is to run pipeline jobs against certain branches within my repository. However, the initial fetch seems to timeout. As you can see from the attached log, the issue appears to arise from a 10 minute timeout issue. However, if you see the below screen shots, I have all of my timeouts set to 180minutes.

      In the attached tickets there were known issues with the 10 minute git timeout that were fixed by allowing users access to overrides as I have shown in the attached images. It appears as though these overrides are not being honored for some reason within the MultibranchPipeline project.

      I have used the settings described here on a smaller repo and it was able to properly index the branches.

      As a work around, I also tried only using the overrides as advanced settings for checkout/fetch behavior. This did not work either, unfortunately.

          [JENKINS-38973] Pipeline initial checkout of Jenkinsfile not honoring git timeout

          Hi, 

          g_doc2080 please check this link: https://stackoverflow.com/a/37133464 

          Antonio Asenjo added a comment - Hi,  g_doc2080  please check this link:  https://stackoverflow.com/a/37133464  

          Hi,

          I second g_doc2080's comment. Our git repository is close to 5GB in size and the local git repository is not being used at all when running "Branch Indexing".

          aseg: thanks for the link but setting the timeout option higher is not really a long term solution since we have a large number of pipelines with all require branch indexing meaning the 5GB will be duplicated many times on the master.

          I too would like to see a solution for this. I will have a look into the code when I have time but I don't know when this will be.

           

          Steve Boardwell added a comment - Hi, I second g_doc2080 's comment. Our git repository is close to 5GB in size and the local git repository is not being used at all when running "Branch Indexing". aseg : thanks for the link but setting the timeout option higher is not really a long term solution since we have a large number of pipelines with all require branch indexing meaning the 5GB will be duplicated many times on the master. I too would like to see a solution for this. I will have a look into the code when I have time but I don't know when this will be.  

          Not exactly the same, but we have a similar problem with the shallow clone option: the plugin doesn't seem to take into account either (no --depth=xx on the git fetch command line)

          Sebastien Hoareau added a comment - Not exactly the same, but we have a similar problem with the shallow clone option: the plugin doesn't seem to take into account either (no --depth=xx on the git fetch command line)

          Rohit Gagan added a comment -

          Any resolution to this issue. Still facing the same problem

          Rohit Gagan added a comment - Any resolution to this issue. Still facing the same problem

          Mark Waite added a comment - - edited

          No rohit_gagan, the best work around is still the stackoverflow article referenced by aseg. Since branch indexing happens on the master and uses a known cache, it should create only a single copy of the large repository, then it will reuse that for additional operations.

          Mark Waite added a comment - - edited No rohit_gagan , the best work around is still the stackoverflow article referenced by aseg . Since branch indexing happens on the master and uses a known cache, it should create only a single copy of the large repository, then it will reuse that for additional operations.

          Rohit Gagan added a comment -

          markewaite. Tried adding that, this still takes timeout as 10 min

           

          Started
          [Wed May 22 11:55:15 EDT 2019] Starting branch indexing...
          > git --version # timeout=10

          Rohit Gagan added a comment - markewaite . Tried adding that, this still takes timeout as 10 min   Started [Wed May 22 11:55:15 EDT 2019] Starting branch indexing... > git --version # timeout=10

          Rohit Gagan added a comment -

          markewaite only option i see is change in git plugin code then to increase the default timeout. What you suggest?

           

          Rohit Gagan added a comment - markewaite only option i see is change in git plugin code then to increase the default timeout. What you suggest?  

          Rohit Gagan added a comment -

          markewaite. Thanks. This works. i was adding the parameter at the wrong end

          Rohit Gagan added a comment - markewaite . Thanks. This works. i was adding the parameter at the wrong end

          Joel added a comment -

          I have this issue too (index-pack died of signal 15) because our repo is too big (20+GB), even though I gave lightweight checkout enabled.

          I'm wondering why Jenkins is not using git archive as an alternative to checking out the whole repository.

          The current method could still be kept as a fallback if the remote has not enabled that option (like for GitHub, for example)

          Joel added a comment - I have this issue too ( index-pack died of signal 15 ) because our repo is too big (20+GB), even though I gave lightweight checkout enabled. I'm wondering why Jenkins is not using git archive  as an alternative to checking out the whole repository. The current method could still be kept as a fallback if the remote has not enabled that option (like for GitHub , for example)

          Mark Waite added a comment -

          glatapoui refer to "Git in the Large - Tips and Tricks" for recommendations of techniques you can use to reduce the challenges of large git repositories. The techniques that have been most effective for me have included (in order of effectiveness):

          1. Use git large file support rather than storing large files directly in git
          2. Place a reference copy of the repository on the agent that is receiving the large repository and update it periodically (weekly, monthly)
          3. Narrow the refspec used in the job definition so that only the necessary branches are fetched from the remote to the local repository (can be a very significant win if there are many branches in the repository and the content of the branches is mostly distinct from other branches)
          4. Use shallow clone to limit the history depth that is retrieved (sometimes this is known to place too heavy a load on the remote server and should not be used)
          5. Use sparse checkout to reduce the disc use for the agent checkout

          I don't plan to implement git archive support in the git plugin.

          Mark Waite added a comment - glatapoui refer to "Git in the Large - Tips and Tricks" for recommendations of techniques you can use to reduce the challenges of large git repositories. The techniques that have been most effective for me have included (in order of effectiveness): Use git large file support rather than storing large files directly in git Place a reference copy of the repository on the agent that is receiving the large repository and update it periodically (weekly, monthly) Narrow the refspec used in the job definition so that only the necessary branches are fetched from the remote to the local repository (can be a very significant win if there are many branches in the repository and the content of the branches is mostly distinct from other branches) Use shallow clone to limit the history depth that is retrieved (sometimes this is known to place too heavy a load on the remote server and should not be used) Use sparse checkout to reduce the disc use for the agent checkout I don't plan to implement git archive support in the git plugin.

            Unassigned Unassigned
            magic David Johnson
            Votes:
            26 Vote for this issue
            Watchers:
            34 Start watching this issue

              Created:
              Updated: