Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-21914

Intermittent Git(hub) polling errors on Windows

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      I haven't yet seen this come up when a build checks out code from Github, but it does happen intermittently when Jenkins polls Github for changes. I've pasted the text that appears on the "Git Polling Log" page at the end of this description. Are there any things I can do to get more and/or better output to help with debugging?

      Thank you.

      Started on Feb 21, 2014 11:09:10 AM
      Using strategy: Default
      [poll] Last Built Revision: Revision 4e9e870ddaabd828a530eb48669fe548e63f3ed2 (origin/master)
      using .gitcredentials to set credentials
      Could not remove the credential section from the git configuration
      FATAL: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
      stdout: 
      stderr: error: could not lock config file .git/config: File exists
      
      hudson.util.IOException2: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
      stdout: 
      stderr: error: could not lock config file .git/config: File exists
      
      	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:457)
      	at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:356)
      	at hudson.scm.SCM.poll(SCM.java:373)
      	at hudson.model.AbstractProject._poll(AbstractProject.java:1581)
      	at hudson.model.AbstractProject.poll(AbstractProject.java:1490)
      	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:462)
      	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:491)
      	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
      	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
      	at java.util.concurrent.FutureTask.run(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.lang.Thread.run(Unknown Source)
      Caused by: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
      stdout: 
      stderr: error: could not lock config file .git/config: File exists
      
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1173)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1150)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1146)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1142)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1039)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:988)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:979)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:1526)
      	at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:487)
      	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:455)
      	... 13 more
      Done. Took 0.82 sec
      No changes
      

        Attachments

          Activity

          cswarren cswarren created issue -
          cswarren cswarren made changes -
          Field Original Value New Value
          Description I haven't yet seen this come up when a build checks out code from Github, but it does happen intermittently when Jenkins polls Github for changes. I've pasted the text that appears on the "Git Polling Log" page at the end of this description. Are there any things I can do to get more and/or better output to help with debugging?

          Thank you.


          Started on Feb 21, 2014 11:09:10 AM
          Using strategy: Default
          [poll] Last Built Revision: Revision 4e9e870ddaabd828a530eb48669fe548e63f3ed2 (origin/master)
          using .gitcredentials to set credentials
          Could not remove the credential section from the git configuration
          FATAL: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
          stdout:
          stderr: error: could not lock config file .git/config: File exists

          hudson.util.IOException2: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
          stdout:
          stderr: error: could not lock config file .git/config: File exists

          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:457)
          at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:356)
          at hudson.scm.SCM.poll(SCM.java:373)
          at hudson.model.AbstractProject._poll(AbstractProject.java:1581)
          at hudson.model.AbstractProject.poll(AbstractProject.java:1490)
          at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:462)
          at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:491)
          at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
          at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
          at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at java.lang.Thread.run(Unknown Source)
          Caused by: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
          stdout:
          stderr: error: could not lock config file .git/config: File exists

          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1173)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1150)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1146)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1142)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1039)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:988)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:979)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:1526)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:487)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:455)
          ... 13 more
          Done. Took 0.82 sec
          No changes
          I haven't yet seen this come up when a build checks out code from Github, but it does happen intermittently when Jenkins polls Github for changes. I've pasted the text that appears on the "Git Polling Log" page at the end of this description. Are there any things I can do to get more and/or better output to help with debugging?

          Thank you.

          {noformat}
          Started on Feb 21, 2014 11:09:10 AM
          Using strategy: Default
          [poll] Last Built Revision: Revision 4e9e870ddaabd828a530eb48669fe548e63f3ed2 (origin/master)
          using .gitcredentials to set credentials
          Could not remove the credential section from the git configuration
          FATAL: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
          stdout:
          stderr: error: could not lock config file .git/config: File exists

          hudson.util.IOException2: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
          stdout:
          stderr: error: could not lock config file .git/config: File exists

          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:457)
          at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:356)
          at hudson.scm.SCM.poll(SCM.java:373)
          at hudson.model.AbstractProject._poll(AbstractProject.java:1581)
          at hudson.model.AbstractProject.poll(AbstractProject.java:1490)
          at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:462)
          at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:491)
          at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
          at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
          at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at java.lang.Thread.run(Unknown Source)
          Caused by: hudson.plugins.git.GitException: Command "git config --local credential.helper store --file=\"C:\Windows\TEMP\git4966117369976036349.credentials\"" returned status code 255:
          stdout:
          stderr: error: could not lock config file .git/config: File exists

          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1173)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1150)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1146)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1142)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1039)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:988)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:979)
          at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:1526)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:487)
          at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:455)
          ... 13 more
          Done. Took 0.82 sec
          No changes
          {noformat}
          cswarren cswarren made changes -
          Environment I've got two machines where this happens. One is Windows 2003 Server, with Jenkins 1.551, git-client 1.6.3, and git 2.0; the installed Git client is msysgit 1.8.1.2-preview20130201. The other machine is Windows 7, with Jenkins 1.551, git-client 1.6.3, git-client 1.6.3, and git 2.0.2; the installed Git client is 1.8.5.2.msysgit.0. The systems have >10 jobs each, polling HTTPS URLs to private repositories on Github, and using global credentials stored in Jenkins. I've got two machines where this happens. One is Windows 2003 Server, with Jenkins 1.551, git-client 1.6.3, and git 2.0; the installed Git client is msysgit 1.8.1.2-preview20130201. The other machine is Windows 7, with Jenkins 1.551, git-client 1.6.3, and git 2.0.2; the installed Git client is 1.8.5.2.msysgit.0. The systems have >10 jobs each, polling HTTPS URLs to private repositories on Github, and using global credentials stored in Jenkins.
          Hide
          markewaite Mark Waite added a comment -

          Since you're on a Windows machine, I suspect that a process is holding that file open, and while the file remains open the Windows file system refuses to delete the file.

          You could check:

          Show
          markewaite Mark Waite added a comment - Since you're on a Windows machine, I suspect that a process is holding that file open, and while the file remains open the Windows file system refuses to delete the file. You could check: Is there a git.exe process still running on the machine? If there is and you kill that process, does it resolve the issue? Is an open file reported by the File Leak Detector plugin ( https://wiki.jenkins-ci.org/display/JENKINS/File+Leak+Detector+Plugin )?
          Hide
          kmart216 David Humeniuk added a comment -

          I have the same issue. There are no git.exe processes running (I'm assuming that the polling would have to run on the master, that's what I checked). git.exe will show up in the task manager, but quickly disappear. When this happens, it is intermittent. Is it possible that Jenkins is trying to use the same file for multiple jobs or something? I tried installing the File Leak Detector plug-in, but got this error when I tried to activate:

          javax.servlet.ServletException: java.lang.Error: Failed to activate file leak detector: Connecting to 0
          Exception in thread "main" java.lang.reflect.InvocationTargetException
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:601)
          	at org.kohsuke.file_leak_detector.Main.run(Main.java:50)
          	at org.kohsuke.file_leak_detector.Main.main(Main.java:35)
          Caused by: java.io.IOException: no such process
          	at sun.tools.attach.WindowsVirtualMachine.openProcess(Native Method)
          	at sun.tools.attach.WindowsVirtualMachine.<init>(WindowsVirtualMachine.java:56)
          	at sun.tools.attach.WindowsAttachProvider.attachVirtualMachine(WindowsAttachProvider.java:69)
          	at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:213)
          	... 6 more
          

          It probably also noteworthy that I'm using this with the Stash Webhooks to Jenkins plug-in (https://marketplace.atlassian.com/plugins/com.nerdwin15.stash-stash-webhook-jenkins). That plug-in will trigger Jenkins to check for branches to build and I have several Jenkins jobs, one job for each development branch (plus several jobs for the master).

          Show
          kmart216 David Humeniuk added a comment - I have the same issue. There are no git.exe processes running (I'm assuming that the polling would have to run on the master, that's what I checked). git.exe will show up in the task manager, but quickly disappear. When this happens, it is intermittent. Is it possible that Jenkins is trying to use the same file for multiple jobs or something? I tried installing the File Leak Detector plug-in, but got this error when I tried to activate: javax.servlet.ServletException: java.lang.Error: Failed to activate file leak detector: Connecting to 0 Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.kohsuke.file_leak_detector.Main.run(Main.java:50) at org.kohsuke.file_leak_detector.Main.main(Main.java:35) Caused by: java.io.IOException: no such process at sun.tools.attach.WindowsVirtualMachine.openProcess(Native Method) at sun.tools.attach.WindowsVirtualMachine.<init>(WindowsVirtualMachine.java:56) at sun.tools.attach.WindowsAttachProvider.attachVirtualMachine(WindowsAttachProvider.java:69) at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:213) ... 6 more It probably also noteworthy that I'm using this with the Stash Webhooks to Jenkins plug-in ( https://marketplace.atlassian.com/plugins/com.nerdwin15.stash-stash-webhook-jenkins ). That plug-in will trigger Jenkins to check for branches to build and I have several Jenkins jobs, one job for each development branch (plus several jobs for the master).
          Hide
          markewaite Mark Waite added a comment -

          File leak detector does not work on Windows (unfortunately). It depends on some specific tools from Linux.

          It is possible (and believed) that the same file is inadvertently being used for multiple jobs. One instance of that condition is described in JENKINS-22039. This is a file of a different name, but seems to have a similar behavior.

          Show
          markewaite Mark Waite added a comment - File leak detector does not work on Windows (unfortunately). It depends on some specific tools from Linux. It is possible (and believed) that the same file is inadvertently being used for multiple jobs. One instance of that condition is described in JENKINS-22039 . This is a file of a different name, but seems to have a similar behavior.
          Hide
          kmart216 David Humeniuk added a comment -

          Is it possible the same workaround of enabling "Force polling using workspace" avoid this issue?

          Show
          kmart216 David Humeniuk added a comment - Is it possible the same workaround of enabling "Force polling using workspace" avoid this issue?
          Hide
          markewaite Mark Waite added a comment -

          It certainly seems possible that it might avoid the problem. Have you checked to see if that change avoids the problem?

          Show
          markewaite Mark Waite added a comment - It certainly seems possible that it might avoid the problem. Have you checked to see if that change avoids the problem?
          Hide
          kmart216 David Humeniuk added a comment -

          I've set the property on a few jobs as a test. I will monitor and let you what I find.

          Thanks,
          Dave

          Show
          kmart216 David Humeniuk added a comment - I've set the property on a few jobs as a test. I will monitor and let you what I find. Thanks, Dave
          Hide
          ryanmalgren Ryan Malgren added a comment -

          I had this problem and just as the error message said there was a file called config.lock that needed to be deleted from .git folder.

          The file was located in a path like this on my Jenkins server:
          C:\GitRepo\master\<RepoName>\.git\config.lock

          Once I deleted that file, my build was happy again.

          Show
          ryanmalgren Ryan Malgren added a comment - I had this problem and just as the error message said there was a file called config.lock that needed to be deleted from .git folder. The file was located in a path like this on my Jenkins server: C:\GitRepo\master\<RepoName>\.git\config.lock Once I deleted that file, my build was happy again.
          Hide
          markewaite Mark Waite added a comment -

          Closing this since I've been unable to duplicate this bug report, and there are steps which seem to resolve the symptom if it happens.

          Show
          markewaite Mark Waite added a comment - Closing this since I've been unable to duplicate this bug report, and there are steps which seem to resolve the symptom if it happens.
          markewaite Mark Waite made changes -
          Resolution Cannot Reproduce [ 5 ]
          Status Open [ 1 ] Resolved [ 5 ]
          markewaite Mark Waite made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 153915 ] JNJira + In-Review [ 207438 ]

            People

            Assignee:
            ndeloof Nicolas De Loof
            Reporter:
            cswarren cswarren
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: