Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22400

Git timeout on other actions than fetch/clone

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • git-client-plugin
    • None

      We have issues with jobs that fail on git timeout. Current git-client version allow to configure timeout for clone and fetch command. However, I see issues with 'git clean' command also. The problem mainly occurs on our windows slaves. Heavy load on virtual machines tend affect file system actions...

      I would like to be able to set the default git timeout (globally if possible).

      Not really sure if this is a bug or improvement.

          [JENKINS-22400] Git timeout on other actions than fetch/clone

          Code changed in jenkins
          User: Nicolas De Loof
          Path:
          src/main/java/hudson/plugins/git/GitSCMBackwardCompatibility.java
          src/main/java/hudson/plugins/git/extensions/impl/SubmoduleOption.java
          src/main/resources/hudson/plugins/git/extensions/impl/CloneOption/config.groovy
          src/main/resources/hudson/plugins/git/extensions/impl/SubmoduleOption/config.groovy
          src/main/resources/hudson/plugins/git/extensions/impl/SubmoduleOption/help-timeout.html
          http://jenkins-ci.org/commit/git-plugin/344523babf34fda71c7c1e52e69d04d1b571cb79
          Log:
          [FIXED JENKINS-22400] option to set submodules update timeout

          Cherry pick from 7dab96f8b5b1ea95e3a92123e6424376d7fa1036

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: src/main/java/hudson/plugins/git/GitSCMBackwardCompatibility.java src/main/java/hudson/plugins/git/extensions/impl/SubmoduleOption.java src/main/resources/hudson/plugins/git/extensions/impl/CloneOption/config.groovy src/main/resources/hudson/plugins/git/extensions/impl/SubmoduleOption/config.groovy src/main/resources/hudson/plugins/git/extensions/impl/SubmoduleOption/help-timeout.html http://jenkins-ci.org/commit/git-plugin/344523babf34fda71c7c1e52e69d04d1b571cb79 Log: [FIXED JENKINS-22400] option to set submodules update timeout Cherry pick from 7dab96f8b5b1ea95e3a92123e6424376d7fa1036

          Mark Waite added a comment -

          Timeout on clean is not yet implemented

          Mark Waite added a comment - Timeout on clean is not yet implemented

          Mark Waite added a comment -

          Timeout on submodule update is implemented in git plugin 2.3 and in git plugin 2.2.8. Timeout on clean is not yet implemented.

          Mark Waite added a comment - Timeout on submodule update is implemented in git plugin 2.3 and in git plugin 2.2.8. Timeout on clean is not yet implemented.

          Robert Beddow added a comment -

          This appears to only work on git fetch for me, not for git checkout, I am using git plugin v2.3. From the comments above it sounds like it should work with checkout, is that correct?

          Robert Beddow added a comment - This appears to only work on git fetch for me, not for git checkout, I am using git plugin v2.3. From the comments above it sounds like it should work with checkout, is that correct?

          Mark Waite added a comment -

          robertbw can you explain what you mean by "git fetch" vs. "git checkout"?

          When I think of "git fetch", I think you mean the timeout which is controlled by the setting in the "Advanced clone behaviours" field labeled "Timeout (in minutes) for clone and fetch operations".

          When I think of "git checkout", I think you mean the timeout which is controlled by the setting in "Advanced checkout behaviours" field labeled "Timeout (in minutes) for checkout operation".

          Is that what you mean in those two cases, or did you possibly miss setting the "Timeout (in minutes) for checkout operation"?

          Mark Waite added a comment - robertbw can you explain what you mean by "git fetch" vs. "git checkout"? When I think of "git fetch", I think you mean the timeout which is controlled by the setting in the "Advanced clone behaviours" field labeled "Timeout (in minutes) for clone and fetch operations". When I think of "git checkout", I think you mean the timeout which is controlled by the setting in "Advanced checkout behaviours" field labeled "Timeout (in minutes) for checkout operation". Is that what you mean in those two cases, or did you possibly miss setting the "Timeout (in minutes) for checkout operation"?

          Robert Beddow added a comment -

          Gotcha - I completely missed that. As you say, I thought the clone&fetch button included the checkout too. Didn't read it carefully. Thanks for helping out.

          Robert Beddow added a comment - Gotcha - I completely missed that. As you say, I thought the clone&fetch button included the checkout too. Didn't read it carefully. Thanks for helping out.

          trejkaz added a comment -

          Is this going to be added for reset as well?

           > git reset --hard # timeout=10
          ERROR: Timeout after 10 minutes

          trejkaz added a comment - Is this going to be added for reset  as well? > git reset --hard # timeout=10 ERROR: Timeout after 10 minutes

          Mark Waite added a comment -

          trejkaz I am not planning to add timeout adjustment support for git reset --hard. I'm impressed that you've found a case where it requires more than 10 minutes to complete. Can you provide more details on the environment where you see git reset --hard require more than 10 minutes?

          I assume it is a large git repository accessed over a network file system with many changes to tracked files in the working directory. How large is the repository? How widespread are the changes? How does the file system perform in general?

          Mark Waite added a comment - trejkaz I am not planning to add timeout adjustment support for git reset --hard . I'm impressed that you've found a case where it requires more than 10 minutes to complete. Can you provide more details on the environment where you see git reset --hard require more than 10 minutes? I assume it is a large git repository accessed over a network file system with many changes to tracked files in the working directory. How large is the repository? How widespread are the changes? How does the file system perform in general?

          trejkaz added a comment - - edited

          It's a large repo but the filesystem is local.

          The only thing could be that Jenkins is often introducing contention with itself - when it scans a repo for new branches to build, it immediately kicks off builds for the branches it finds, which all require checkouts done on the master, which then all run at the same time, making all of them run slower.

          As far as changes to tracked files, there wouldn't be any changes done by ourselves, but Jenkins might have done something as part of doing the merge between master and whatever branch it's building.

           

          trejkaz added a comment - - edited It's a large repo but the filesystem is local. The only thing could be that Jenkins is often introducing contention with itself - when it scans a repo for new branches to build, it immediately kicks off builds for the branches it finds, which all require checkouts done on the master, which then all run at the same time, making all of them run slower. As far as changes to tracked files, there wouldn't be any changes done by ourselves, but Jenkins might have done something as part of doing the merge between master and whatever branch it's building.  

          Marat Radchenko added a comment - - edited

          I'm impressed that you've found a case where it requires more than 10 minutes to complete.

          Large repo + interrupted checkout during previous build == almost all I/O happens during git reset --hard. If you add LFS to the mix, that I/O also involves transferring files over network.

          Simpler way to reproduce: just remove everything from working copy, but keep .git it in. Voila, you've got a massive git reset --hard for next build in the same working copy.

          Marat Radchenko added a comment - - edited I'm impressed that you've found a case where it requires more than 10 minutes to complete. Large repo + interrupted checkout during previous build == almost all I/O happens during git reset --hard . If you add LFS to the mix, that I/O also involves transferring files over network. Simpler way to reproduce: just remove everything from working copy, but keep .git it in. Voila, you've got a massive git reset --hard for next build in the same working copy.

            Unassigned Unassigned
            jonas_olsson Jonas Olsson
            Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: