Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24809

Github-Jenkins Trigger issue with slave

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I have a jenkins instance which has a windows slave machine attached. I also have a job running which resides in github and I have the trigger - "Build when a change is pushed to GitHub" set.

      as I push a change to github, there is no build triggered on jenkins. I check and see that the payload does get tranferred to jenkins in the "Github hook log" but I get the following error:

      Using strategy: Default
      [poll] Last Built Revision: Revision ce6a183e834a3e31afa0eb83a4418b0619c8642b (origin/master)
      > "C:\Program Files (x86)\Git\cmd\git.exe" ls-remote -h https://xx/xx/xx master # timeout=10
      FATAL: hudson.plugins.git.GitException: Error performing command: "C:\Program Files (x86)\Git\cmd\git.exe" ls-remote -h https://xx/xx/xx master
      hudson.util.IOException2: hudson.plugins.git.GitException: Error performing command: "C:\Program Files (x86)\Git\cmd\git.exe" ls-remote -h https://xx/xx/xx master
      at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:462)
      at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:357)
      at hudson.scm.SCM.poll(SCM.java:374)
      at hudson.model.AbstractProject._poll(AbstractProject.java:1428)
      at hudson.model.AbstractProject.poll(AbstractProject.java:1331)
      at com.cloudbees.jenkins.GitHubPushTrigger$1.runPolling(GitHubPushTrigger.java:73)
      at com.cloudbees.jenkins.GitHubPushTrigger$1.run(GitHubPushTrigger.java:98)
      at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
      at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:744)
      Caused by: hudson.plugins.git.GitException: Error performing command: "C:\Program Files (x86)\Git\cmd\git.exe" ls-remote -h https://xx/xx/xx master
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1444)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1225)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1138)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1129)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:2059)
      at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:495)
      at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:460)
      ... 13 more
      Caused by: java.io.IOException: Cannot run program ""C:\Program Files (x86)\Git\cmd\git.exe"": error=2, No such file or directory

      Basically complains about not being able to run the command C:\Program Files (x86)\Git\cmd\git.exe" ls-remote -h https://xx/xx/xx master

      But as I manually go into my slave machine and run the command, it works fine.

      Please note that 1. I have made the adjustment where jenkins slave service logs in as admin so that it has all permissions. 2. My default jenkins server is a linux box and the slave machine is a windows box. Obviously both have different git executable path. So I am wondering if jenkins runs this command on my linux box for some reason.

      I do have the project setup to run on the windows slave box only and that works fine as it builds fine(so it is able to find the git executable for the job)

      Any pointers?

      Also does anyone know how github plugin for jenkins works with slave machines(does it look for git in the slave or does it go to where jenkins resides, as that might be the source, though have no idea what the solution for that weird case would be?)

        Attachments

          Activity

          kaulk Kuber Kaul created issue -
          kaulk Kuber Kaul made changes -
          Field Original Value New Value
          Assignee Kuber Kaul [ kaulk ] R. Tyler Croy [ rtyler ]
          kaulk Kuber Kaul made changes -
          Assignee R. Tyler Croy [ rtyler ] Kohsuke Kawaguchi [ kktest10 ]
          Hide
          rtyler R. Tyler Croy added a comment -

          I'm going to move this into the JENKINS project, this doesn't appear to have anything to do with the project's own infrastructure

          Show
          rtyler R. Tyler Croy added a comment - I'm going to move this into the JENKINS project, this doesn't appear to have anything to do with the project's own infrastructure
          rtyler R. Tyler Croy made changes -
          Component/s github [ 15896 ]
          Component/s github [ 18925 ]
          Key INFRA-140 JENKINS-24809
          Project Infrastructure [ 10301 ] Jenkins [ 10172 ]
          Workflow jira [ 158030 ] JNJira [ 158038 ]
          Hide
          kaulk Kuber Kaul added a comment -

          Thanks!

          Show
          kaulk Kuber Kaul added a comment - Thanks!
          Hide
          kaulk Kuber Kaul added a comment -

          I tried a work around of adding the "Force polling using workspace" as an "Additional behaviour" in my git SCM definition of the windows job.

          If the poll runs on the Linux node (the master) but somehow decides it needs to use the Windows git configuration to poll the master, that might cause the Linux master to attempt to execute git from the Windows git location, instead of the Linux git location(which is what is happening in my project). That would be a bug, and I was surprised at this bug, since there are many installations of Jenkins git which use a wide mix of Linux and Windows slaves.

          I'm was not aware of such a bug, but since the "Force polling using workspace" resolved the problem for you, it may hint that is an area for more investigation.

          Show
          kaulk Kuber Kaul added a comment - I tried a work around of adding the "Force polling using workspace" as an "Additional behaviour" in my git SCM definition of the windows job. If the poll runs on the Linux node (the master) but somehow decides it needs to use the Windows git configuration to poll the master, that might cause the Linux master to attempt to execute git from the Windows git location, instead of the Linux git location(which is what is happening in my project). That would be a bug, and I was surprised at this bug, since there are many installations of Jenkins git which use a wide mix of Linux and Windows slaves. I'm was not aware of such a bug, but since the "Force polling using workspace" resolved the problem for you, it may hint that is an area for more investigation.
          Hide
          danielbeck Daniel Beck added a comment -

          Seems to be more of a Git plugin issue> Could you provide all relevant Jenkins/plugin version numbers?

          Show
          danielbeck Daniel Beck added a comment - Seems to be more of a Git plugin issue> Could you provide all relevant Jenkins/plugin version numbers?
          danielbeck Daniel Beck made changes -
          Component/s git [ 15543 ]
          Assignee Kohsuke Kawaguchi [ kktest10 ]
          Hide
          kaulk Kuber Kaul added a comment -

          Github plugin - 1.9.1
          Git plugin - 2.2.6
          Jenkins - 1.580

          It does seem like github plugin bug.

          Show
          kaulk Kuber Kaul added a comment - Github plugin - 1.9.1 Git plugin - 2.2.6 Jenkins - 1.580 It does seem like github plugin bug.
          Hide
          markewaite Mark Waite added a comment -

          There are certainly many installations worldwide which use a wide mix of Windows and Linux computers, and which successfully poll whether or not "Force polling workspace" is set.

          You may have a configuration error in your Jenkins setup. Some things you might try as a way to test that theory:

          1. Configure a freestyle job with Git as scm from a repository accessible from all your machines (maybe https://github.com/jenkinsci/git-client-plugin.git)
          2. Configure that job to "Force polling using workspace" and configure it to poll once a minute
          3. Configure the job to limit which node can run the job, limit it to the master node
          4. Read the polling log to confirm it was successful
          5. Configure the job to limit which node can run the job, limit it to another node
          6. Wait for it to poll from that node
          7. Read the polling log to see if the polling commands are successful from that node
          8. Iterate through each of your nodes, confirming that each is able to poll with git

          If one of the nodes is discovered that it cannot poll with git, then you can review the configuration of that node, looking to see if it has incorrectly declared the location of the git program, or if it is incorrectly using the git location of the master node.

          Show
          markewaite Mark Waite added a comment - There are certainly many installations worldwide which use a wide mix of Windows and Linux computers, and which successfully poll whether or not "Force polling workspace" is set. You may have a configuration error in your Jenkins setup. Some things you might try as a way to test that theory: Configure a freestyle job with Git as scm from a repository accessible from all your machines (maybe https://github.com/jenkinsci/git-client-plugin.git ) Configure that job to "Force polling using workspace" and configure it to poll once a minute Configure the job to limit which node can run the job, limit it to the master node Read the polling log to confirm it was successful Configure the job to limit which node can run the job, limit it to another node Wait for it to poll from that node Read the polling log to see if the polling commands are successful from that node Iterate through each of your nodes, confirming that each is able to poll with git If one of the nodes is discovered that it cannot poll with git, then you can review the configuration of that node, looking to see if it has incorrectly declared the location of the git program, or if it is incorrectly using the git location of the master node.
          Hide
          markewaite Mark Waite added a comment -

          After two weeks with no response to my observation that I'm unable to duplicate this bug, I'm closing it as "Cannot Reproduce". I run a Jenkins instance with a Linux master and several Windows slaves and do not see this problem.

          Show
          markewaite Mark Waite added a comment - After two weeks with no response to my observation that I'm unable to duplicate this bug, I'm closing it as "Cannot Reproduce". I run a Jenkins instance with a Linux master and several Windows slaves and do not see this problem.
          markewaite Mark Waite made changes -
          Resolution Cannot Reproduce [ 5 ]
          Status Open [ 1 ] Resolved [ 5 ]
          markewaite Mark Waite made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          kbaltrinic Kenneth Baltrinic added a comment -

          Mark Waite I would like to re-open this issue. We are encountering the exact same behavior, though our setup is a bit different. We are not using git hooks in our case, simple SCM Polling. As with the originally reported situation, this is happening with a Linux master an multiple (8) windows slaves. All slaves are presenting with the same issue. Setting "Force polling workspace" also works as a workaround in our case.

          We are seeing this with Jenkins 1.609.1, Git 2.4, Git client 1.16.1

          Below is our stack trace, though it does not seem to differ materially from the one in the original report:

          Started on Aug 4, 2015 12:37:01 PM
          Polling SCM changes on hq-jenwin-007
          Using strategy: Default
          [poll] Last Built Revision: Revision c61ffa28bf6375f1907d49cc7c5ad282c32c5f1c (refs/remotes/origin/prod)
          using GIT_SSH to set credentials Credential ID: ******
           > C:\Program Files (x86)\Git\cmd\git.exe --version # timeout=10
           > C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh://git@stash:7999/**/**.git # timeout=10
          FATAL: hudson.plugins.git.GitException: Error performing command: C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh://git@stash:7999/**/**.git
          hudson.util.IOException2: hudson.plugins.git.GitException: Error performing command: C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh://git@stash:7999/**/**.git
            at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:529)
            at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:380)
            at hudson.scm.SCM.poll(SCM.java:397)
            at org.jenkinsci.plugins.multiplescms.MultiSCM.compareRemoteRevisionWith(MultiSCM.java:100)
            at hudson.scm.SCM.poll(SCM.java:397)
            at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1467)
            at hudson.model.AbstractProject._poll(AbstractProject.java:1438)
            at hudson.model.AbstractProject.poll(AbstractProject.java:1349)
            at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510)
            at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539)
            at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
          Caused by: hudson.plugins.git.GitException: Error performing command: C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh://git@stash:7999/**/**.git
            at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1598)
            at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1379)
            at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1277)
            at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1268)
            at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:2327)
            at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:583)
            at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:527)
            ... 15 more
          Caused by: java.io.IOException: Cannot run program "C:\Program Files (x86)\Git\cmd\git.exe": error=2, No such file or directory
            at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
            at hudson.Proc$LocalProc.<init>(Proc.java:244)
            at hudson.Proc$LocalProc.<init>(Proc.java:216)
            at hudson.Launcher$LocalLauncher.launch(Launcher.java:803)
            at hudson.Launcher$ProcStarter.start(Launcher.java:381)
            at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1587)
            ... 21 more
          Caused by: java.io.IOException: error=2, No such file or directory
            at java.lang.UNIXProcess.forkAndExec(Native Method)
            at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
            at java.lang.ProcessImpl.start(ProcessImpl.java:134)
            at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
            ... 26 more
          Done. Took 50 ms
          No changes
          
          Show
          kbaltrinic Kenneth Baltrinic added a comment - Mark Waite I would like to re-open this issue. We are encountering the exact same behavior, though our setup is a bit different. We are not using git hooks in our case, simple SCM Polling. As with the originally reported situation, this is happening with a Linux master an multiple (8) windows slaves. All slaves are presenting with the same issue. Setting "Force polling workspace" also works as a workaround in our case. We are seeing this with Jenkins 1.609.1, Git 2.4, Git client 1.16.1 Below is our stack trace, though it does not seem to differ materially from the one in the original report: Started on Aug 4, 2015 12:37:01 PM Polling SCM changes on hq-jenwin-007 Using strategy: Default [poll] Last Built Revision: Revision c61ffa28bf6375f1907d49cc7c5ad282c32c5f1c (refs/remotes/origin/prod) using GIT_SSH to set credentials Credential ID: ****** > C:\Program Files (x86)\Git\cmd\git.exe --version # timeout=10 > C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh: //git@stash:7999 /**/ **.git # timeout=10 FATAL: hudson.plugins.git.GitException: Error performing command: C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh: //git@stash:7999 /**/ **.git hudson.util.IOException2: hudson.plugins.git.GitException: Error performing command: C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh: //git@stash:7999 /**/ **.git at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:529) at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:380) at hudson.scm.SCM.poll(SCM.java:397) at org.jenkinsci.plugins.multiplescms.MultiSCM.compareRemoteRevisionWith(MultiSCM.java:100) at hudson.scm.SCM.poll(SCM.java:397) at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1467) at hudson.model.AbstractProject._poll(AbstractProject.java:1438) at hudson.model.AbstractProject.poll(AbstractProject.java:1349) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang. Thread .run( Thread .java:745) Caused by: hudson.plugins.git.GitException: Error performing command: C:\Program Files (x86)\Git\cmd\git.exe ls-remote -h ssh: //git@stash:7999 /**/ **.git at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1598) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1379) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1277) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1268) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:2327) at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:583) at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:527) ... 15 more Caused by: java.io.IOException: Cannot run program "C:\Program Files (x86)\Git\cmd\git.exe" : error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at hudson.Proc$LocalProc.<init>(Proc.java:244) at hudson.Proc$LocalProc.<init>(Proc.java:216) at hudson.Launcher$LocalLauncher.launch(Launcher.java:803) at hudson.Launcher$ProcStarter.start(Launcher.java:381) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1587) ... 21 more Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:187) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 26 more Done. Took 50 ms No changes
          Hide
          kbaltrinic Kenneth Baltrinic added a comment -

          I should have added that we recently upgraded from an old version of jenkins where we were not seeing this behavior: Our old system was Jenkins 1.552, Git plugin 1.4 and Git Client plugin 1.6.

          Show
          kbaltrinic Kenneth Baltrinic added a comment - I should have added that we recently upgraded from an old version of jenkins where we were not seeing this behavior: Our old system was Jenkins 1.552, Git plugin 1.4 and Git Client plugin 1.6.
          Hide
          markewaite Mark Waite added a comment -

          I think the problem you're seeing is due to JENKINS-28943 and is resolved in git client plugin 1.18.0. I'm surprised you didn't update to git client plugin 1.18.0 at the same time you updated to git plugin 2.4.0.

          If that does not fix it, then you can also work-around the problem by changing the path to the git executable on those computers from C:\Program Files (x86)\Git\cmd\git.exe to C:\Program Files (x86)\Git\bin\git.exe

          Show
          markewaite Mark Waite added a comment - I think the problem you're seeing is due to JENKINS-28943 and is resolved in git client plugin 1.18.0. I'm surprised you didn't update to git client plugin 1.18.0 at the same time you updated to git plugin 2.4.0. If that does not fix it, then you can also work-around the problem by changing the path to the git executable on those computers from C:\Program Files (x86)\Git\cmd\git.exe to C:\Program Files (x86)\Git\bin\git.exe
          Hide
          kbaltrinic Kenneth Baltrinic added a comment -

          Mark Waite It took me a while get back to this. I have upgraded our git client to 1.18.0 but we are still experiencing the same problem as I detailed above.

          Show
          kbaltrinic Kenneth Baltrinic added a comment - Mark Waite It took me a while get back to this. I have upgraded our git client to 1.18.0 but we are still experiencing the same problem as I detailed above.
          Hide
          markewaite Mark Waite added a comment -

          Kenneth Baltrinic have you tried the alternative I suggested in my previous comment, changing the Jenkins configured path for the git executable from

          C:\Program Files (x86)\Git\cmd\git.exe
          

          to

          C:\Program Files (x86)\Git\bin\git.exe
          
          Show
          markewaite Mark Waite added a comment - Kenneth Baltrinic have you tried the alternative I suggested in my previous comment, changing the Jenkins configured path for the git executable from C:\Program Files (x86)\Git\cmd\git.exe to C:\Program Files (x86)\Git\bin\git.exe
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 158038 ] JNJira + In-Review [ 208069 ]

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            kaulk Kuber Kaul
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: