Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22547

Git timeout setting does not work for checkout

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-plugin
    • Windows (8) (the slow disk access probably makes it more likely to manifest)

      Some git processes error out with ERROR: Timeout after 10 minutes even though longer timeout is configured in the job.

      It affects the `git checkout` operation itself. The fetch operation is OK.

      The log says:

      using GIT_SSH to set credentials jenkins key for git
       > git fetch --tags --progress git@git.company.com:project +refs/heads/*:refs/remotes/origin/* --prune
      Checking out Revision 159bc2b21669bc7b5217341fc8de9cd6b48439b2 (origin/dev/jan.hudec/pu)
       > git config core.sparsecheckout
       > git checkout -f 159bc2b21669bc7b5217341fc8de9cd6b48439b2
      ERROR: Timeout after 10 minutes
      FATAL: Could not checkout null with start point 159bc2b21669bc7b5217341fc8de9cd6b48439b2
      

      When I manually removed the lock and repeated the checkout operation, it indeed took 11 minutes 15 seconds on the node where it failed.

      The global timeout does work, so it's not a blocker anymore. It is, however, rather non-obvious configuration as the -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=30 (or whatever sufficiently large value) option needs to be added to both JVM options of the master and JVM options of all slaves. The master options can only be configured in the servlet container and while the slave options can be configured in node settings (hidden out under "Advanced" button), slaves running as windows service don't take this into account without reinstalling the service.

          [JENKINS-22547] Git timeout setting does not work for checkout

          Mark Waite added a comment -

          Included in git client plugin 1.20.0 released 20 Aug 2016

          Mark Waite added a comment - Included in git client plugin 1.20.0 released 20 Aug 2016

          Balaji Santhanam added a comment - - edited

          markewaite : I am using git plugin 3.0.1 & Git client plugin 2.2.0 and could reproduce the timeout issue with sparse checkout. I tried setting up the timeout in Additional Behaviors >> Timeout=120, Still its not working find below the Jenkins output. Let me know If I am missing anything here.

          Building remotely on JenkinsSlave in workspace /home/balaji/workspace/PublishOverSSH
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repository
           > git config remote.origin.url https://bitbucket.org/test.git # timeout=10
          Fetching upstream changes from https://bitbucket.org/test.git
           > git --version # timeout=10
          using GIT_ASKPASS to set credentials 
           > git fetch --tags --progress https://bitbucket.org/test.git +refs/heads/*:refs/remotes/origin/* # timeout=120
           > git rev-parse refs/remotes/origin/testing^{commit} # timeout=10
           > git rev-parse refs/remotes/origin/origin/testing^{commit} # timeout=10
          Checking out Revision 5d2b54126a0771fc4c205b1c0d4deafd3185d07b (refs/remotes/origin/testing)
           > git config core.sparsecheckout # timeout=10
           > git checkout -f 5d2b54126a0771fc4c205b1c0d4deafd3185d07b
          ERROR: Timeout after 10 minutes
          

          Update : The Advanced Checkout Behavior seem to do the trick for me. So closing the ticket.

          Thanks
          Balaji

          Balaji Santhanam added a comment - - edited markewaite : I am using git plugin 3.0.1 & Git client plugin 2.2.0 and could reproduce the timeout issue with sparse checkout. I tried setting up the timeout in Additional Behaviors >> Timeout=120, Still its not working find below the Jenkins output. Let me know If I am missing anything here. Building remotely on JenkinsSlave in workspace /home/balaji/workspace/PublishOverSSH > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https: //bitbucket.org/test.git # timeout=10 Fetching upstream changes from https: //bitbucket.org/test.git > git --version # timeout=10 using GIT_ASKPASS to set credentials > git fetch --tags --progress https: //bitbucket.org/test.git +refs/heads/*:refs/remotes/origin/* # timeout=120 > git rev-parse refs/remotes/origin/testing^{commit} # timeout=10 > git rev-parse refs/remotes/origin/origin/testing^{commit} # timeout=10 Checking out Revision 5d2b54126a0771fc4c205b1c0d4deafd3185d07b (refs/remotes/origin/testing) > git config core.sparsecheckout # timeout=10 > git checkout -f 5d2b54126a0771fc4c205b1c0d4deafd3185d07b ERROR: Timeout after 10 minutes Update : The Advanced Checkout Behavior seem to do the trick for me. So closing the ticket. Thanks Balaji

          Oded Arbel added a comment - - edited

          I have the same problem - timeout is always "10" and there is no way to change it.

          I've added the advanced checkout behavior and that did not change a thing. Here's the pipeline syntax I'm using:

           

          checkout scm: [$class: 'GitSCM', 
              branches: [[name: '*/master']], 
              doGenerateSubmoduleConfigurations: false, 
              extensions: [[$class: 'CheckoutOption', timeout: 240]], 
              gitTool: 'Default', 
              submoduleCfg: [], 
              userRemoteConfigs: [[
                  credentialsId: '981cbff3-3a30-4a61-be40-99f441ea0559', 
                  url: 'git@server:project.git'
              ]]]
          

          And it looks the same - the logs shows `timeout=10` and it ends with:

          ERROR: Timeout after 10 minutes
          ERROR: Error cloning remote repo 'origin'
          

          Oded Arbel added a comment - - edited I have the same problem - timeout is always "10" and there is no way to change it. I've added the advanced checkout behavior and that did not change a thing. Here's the pipeline syntax I'm using:   checkout scm: [$class: 'GitSCM' , branches: [[name: '*/master' ]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'CheckoutOption' , timeout: 240]], gitTool: 'Default' , submoduleCfg: [], userRemoteConfigs: [[ credentialsId: '981cbff3-3a30-4a61-be40-99f441ea0559' , url: 'git@server:project.git' ]]] And it looks the same - the logs shows `timeout=10` and it ends with: ERROR: Timeout after 10 minutes ERROR: Error cloning remote repo 'origin'

          Oded Arbel added a comment -

          Update, my mistake - should have used advanced clone behavior

          Oded Arbel added a comment - Update, my mistake - should have used advanced clone behavior

          guss77 can you provide an example or link of what you mean by advanced clone behaviour? I'm still getting the 10 minute timeout using the following:

          git branch: '$BRANCH',
          
              credentialsId: 'XXXX', 
              url: 'ssh://git@example.com/myRepo.git', 
              extensions: [[$class: 'CheckoutOption', timeout: 100]]
          

          I tried the `scm checkout` syntax you reference in your first comment and I see the 10 minute timeout just the same as you were describing.

          Additionally, how did you confirm the "advanced behaviour" was working?  Will the log messages begin to display the configured timeout value, or do I just have to test it and hope my network is running slow enough to validate the test?

           

          Luke Lussenden added a comment - guss77 can you provide an example or link of what you mean by advanced clone behaviour? I'm still getting the 10 minute timeout using the following: git branch: '$BRANCH' , credentialsId: 'XXXX' , url: 'ssh: //git@example.com/myRepo.git' , extensions: [[$class: 'CheckoutOption' , timeout: 100]] I tried the `scm checkout` syntax you reference in your first comment and I see the 10 minute timeout just the same as you were describing. Additionally, how did you confirm the "advanced behaviour" was working?  Will the log messages begin to display the configured timeout value, or do I just have to test it and hope my network is running slow enough to validate the test?  

          Oded Arbel added a comment -

          Yes - adding an "advanced clone behavior" and then setting the timeout there solved my problem.

          Oded Arbel added a comment - Yes - adding an "advanced clone behavior" and then setting the timeout there solved my problem.

          Mark Waite added a comment -

          vadivel the bug was fixed over 2 years ago. and is as described by knighttp01 in the comment above your comment that you're reopening the bug.

          It is not enough to say "the same issue occurring". You'll need to provide much more context than "same issue occurring". What have you tried? What is the context where timeouts are not behaving as you expect? What is the log content when the timeout does not behave as you expect? What job type are you using?

          Please gather those details and submit a new bug report, rather than reopening this report.

          There are many, many users that are successfully using extended timeouts to clone and checkout large repositories. I've presented talks at Jenkins World 2016, Jenkins World 2017,at a 2016 online meetup and at a 2017 online meetup that describe techniques to better manage large repositories. All of those talks depend on adjusting timeout values as needed, and they work.

          In addition to those resources, CloudBees support has provided detailed instructions for configuring a reference repository to speed clone operations.

          Mark Waite added a comment - vadivel the bug was fixed over 2 years ago. and is as described by knighttp01 in the comment above your comment that you're reopening the bug. It is not enough to say "the same issue occurring". You'll need to provide much more context than "same issue occurring". What have you tried? What is the context where timeouts are not behaving as you expect? What is the log content when the timeout does not behave as you expect? What job type are you using? Please gather those details and submit a new bug report, rather than reopening this report. There are many, many users that are successfully using extended timeouts to clone and checkout large repositories. I've presented talks at Jenkins World 2016 , Jenkins World 2017 ,at a 2016 online meetup and at a 2017 online meetup that describe techniques to better manage large repositories. All of those talks depend on adjusting timeout values as needed, and they work. In addition to those resources, CloudBees support has provided detailed instructions for configuring a reference repository to speed clone operations.

          yang z added a comment - - edited

          Seems resolved by this way: set timeout by "CloneOption"

          checkout scm: [$class: 'GitSCM', branches: [[name: '*/master']], 
                          doGenerateSubmoduleConfigurations: false, 
                          extensions: [[$class: 'CloneOption', timeout: 240]], // CheckoutOption -> CloneOption                                                 
                          gitTool: 'Default', 
                          submoduleCfg: [], 
                          userRemoteConfigs: [[ credentialsId: '981cbff3-3a30-4a61-be40-99f441ea0559', 
                                          url: 'git@server:project.git' 
                          ]]]
          

          yang z added a comment - - edited Seems resolved by this way: set timeout by "CloneOption" checkout scm: [$class: 'GitSCM' , branches: [[name: '*/master' ]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'CloneOption' , timeout: 240]], // CheckoutOption -> CloneOption gitTool: 'Default' , submoduleCfg: [], userRemoteConfigs: [[ credentialsId: '981cbff3-3a30-4a61-be40-99f441ea0559' , url: 'git@server:project.git' ]]]

          Yarden Bar added a comment -

          Hi,
          We encountered this timeout as well, but we suspect that the git config core.sparsecheckout with out explicit true argument causes this.

          Adding the CloneOption with timeout didn't help in our case.

          Is there a way to configure git config core.sparsecheckout true in the checkout step? I've read the documentation but coudn't figure out how to do so...

           

          Thank you,
          Yarden

          Yarden Bar added a comment - Hi, We encountered this timeout as well, but we suspect that the git config core.sparsecheckout with out explicit true argument causes this. Adding the CloneOption with timeout didn't help in our case. Is there a way to configure git config core.sparsecheckout true in the checkout step? I've read the documentation but coudn't figure out how to do so...   Thank you, Yarden

          Mark Waite added a comment -

          judgedredd it is generally considered bad form to ask an unrelated question in a bug report. It clutters the bug reports and wastes the time of maintainers that are notified when the unrelated comment is added to the bug report. Please don't do that in the future. Use the mailing list or chat questions and answers so that more than a few people are notified and might be able to assist.

          The question interested me enough that I added it to my JENKINS-52746 test case. Refer to that example for details that will allow you to use a sparse checkout path definition in a declarative Pipeline checkout statement.

          If you are using declarative Pipeline, you may also need to `skipDefaultCheckout(true)` in the options section, otherwise the full repository checkout happens implicitly before the first step.

          Mark Waite added a comment - judgedredd it is generally considered bad form to ask an unrelated question in a bug report. It clutters the bug reports and wastes the time of maintainers that are notified when the unrelated comment is added to the bug report. Please don't do that in the future. Use the mailing list or chat questions and answers so that more than a few people are notified and might be able to assist. The question interested me enough that I added it to my JENKINS-52746 test case . Refer to that example for details that will allow you to use a sparse checkout path definition in a declarative Pipeline checkout statement. If you are using declarative Pipeline, you may also need to `skipDefaultCheckout(true)` in the options section, otherwise the full repository checkout happens implicitly before the first step.

            markewaite Mark Waite
            bulb Jan Hudec
            Votes:
            1 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: