Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24467

ssh not found by git because PATH has been unexpectedly changed

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-plugin
    • None
    • Jenkins ver. 1.565.1
      GIT client plugin 1.10.1
      GIT plugin 2.2.5
      GitHub API Plugin 1.56
      GitHub plugin 1.9.1

      Looks like an identical issue to JENKINS-14321, our setup is similar with a debian master and a windows 8 slave. GitHub Hook log is as follows:

      Started on 27-Aug-2014 16:48:45
Polling SCM changes on XXX
Using strategy: Default
[poll] Last Built Revision: Revision a9212bd3ddab1e9f9303be8d7db06cf8b4087b8b (origin/master)
 > git ls-remote -h git@XXX:XXX/XXX.git master # timeout=10
FATAL: hudson.plugins.git.GitException: Command "git ls-remote -h git@XXX:XXX/XXX.git master" returned status code 128:
stdout: 
stderr: error: cannot run ssh: No such file or directory
fatal: unable to fork

hudson.util.IOException2: hudson.plugins.git.GitException: Command "git ls-remote -h git@XXX:XXX/XXX.git master" returned status code 128:
stdout: 
stderr: error: cannot run ssh: No such file or directory
fatal: unable to fork

	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:462)
	at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:357)
	at hudson.scm.SCM.poll(SCM.java:374)
	at org.jenkinsci.plugins.multiplescms.MultiSCM.compareRemoteRevisionWith(MultiSCM.java:92)
	at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:357)
	at hudson.scm.SCM.poll(SCM.java:374)
	at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1449)
	at hudson.model.AbstractProject._poll(AbstractProject.java:1420)
	at hudson.model.AbstractProject.poll(AbstractProject.java:1331)
	at com.cloudbees.jenkins.GitHubPushTrigger$1.runPolling(GitHubPushTrigger.java:73)
	at com.cloudbees.jenkins.GitHubPushTrigger$1.run(GitHubPushTrigger.java:98)
	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: hudson.plugins.git.GitException: Command "git ls-remote -h git@XXX:XXX/XXX.git master" returned status code 128:
stdout: 
stderr: error: cannot run ssh: No such file or directory
fatal: unable to fork

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1407)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1195)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1119)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1110)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(CliGitAPIImpl.java:2004)
	at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:495)
	at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:460)
	... 18 more
Done. Took 20 ms
No changes

          [JENKINS-24467] ssh not found by git because PATH has been unexpectedly changed

          Mark Waite added a comment -

          I'd guess then that the portion which exposes the misconfiguration in your Jenkins job or Jenkins environment to the git plugin is at line 255 and 256 where the comment says:

          // add env contributing actions & values from last build to environment - fixes JENKINS-22009

          That's the key change which was in that commit. As far as I can tell, it is doing "the right thing" by making the environment variables available inside the plugin.

          I think that its discovery of your EnvInject related configuration issue is a happy side effect, not a bug which the git plugin can fix.

          Mark Waite added a comment - I'd guess then that the portion which exposes the misconfiguration in your Jenkins job or Jenkins environment to the git plugin is at line 255 and 256 where the comment says: // add env contributing actions & values from last build to environment - fixes JENKINS-22009 That's the key change which was in that commit. As far as I can tell, it is doing "the right thing" by making the environment variables available inside the plugin. I think that its discovery of your EnvInject related configuration issue is a happy side effect, not a bug which the git plugin can fix.

          Rob Langley added a comment -

          Following the code, it looks like ln:486 in GitSCM.java deicides that it's possible to use fast remote poll.

                  // fast remote polling needs a single branch and an existing last build
        if (!requiresWorkspaceForPolling() && buildData.lastBuild != null && buildData.lastBuild.getMarked() != null) {

            // FIXME this should not be a specific case, but have BuildChooser tell us if it can poll without workspace.

            final EnvVars environment = GitUtils.getPollEnvironment(project, workspace, launcher, listener, false);

            GitClient git = createClient(listener, environment, project, Jenkins.getInstance(), null);

          This then uses getPollEnvironment (and in turn addEnvironmentContributingActionsValues) to get the last build env. This is working correctly as we saw in my previous comments, but it then uses this env on the Jenkins master to do the poll which is why I'm hitting the issue.

          This as least gives a workaround of forcing the polling to use a workspace. I guess EnvInject could be messing with this code but I don't know much about it.

          Rob Langley added a comment - Following the code, it looks like ln:486 in GitSCM.java deicides that it's possible to use fast remote poll. // fast remote polling needs a single branch and an existing last build if (!requiresWorkspaceForPolling() && buildData.lastBuild != null && buildData.lastBuild.getMarked() != null) { // FIXME this should not be a specific case, but have BuildChooser tell us if it can poll without workspace. final EnvVars environment = GitUtils.getPollEnvironment(project, workspace, launcher, listener, false); GitClient git = createClient(listener, environment, project, Jenkins.getInstance(), null); This then uses getPollEnvironment (and in turn addEnvironmentContributingActionsValues) to get the last build env. This is working correctly as we saw in my previous comments, but it then uses this env on the Jenkins master to do the poll which is why I'm hitting the issue. This as least gives a workaround of forcing the polling to use a workspace. I guess EnvInject could be messing with this code but I don't know much about it.

          Daniel Beck added a comment -

          values from last build to environment

          In the case of node specific environment variables (can you tell the difference?), this is just wrong.

          Daniel Beck added a comment - values from last build to environment In the case of node specific environment variables (can you tell the difference?), this is just wrong.

          Rob Langley added a comment -

          Should addEnvironmentContributingActionsValues respect the reuseLastBuildEnv boolean that's passed in to getPollEnvironment:

          diff --git a/src/main/java/hudson/plugins/git/util/GitUtils.java b/src/main/java/hudson/plugins/git/util/GitUtils.java
index 3d41ad3..68669bb 100644
--- a/src/main/java/hudson/plugins/git/util/GitUtils.java
+++ b/src/main/java/hudson/plugins/git/util/GitUtils.java
@@ -254,7 +254,8 @@ public class GitUtils implements Serializable {
         }
 
         // add env contributing actions' values from last build to environment - fixes JENKINS-22009
-        addEnvironmentContributingActionsValues(env, b);
+        if (reuseLastBuildEnv)
+            addEnvironmentContributingActionsValues(env, b);
 
         EnvVars.resolve(env);

          Rob Langley added a comment - Should addEnvironmentContributingActionsValues respect the reuseLastBuildEnv boolean that's passed in to getPollEnvironment: diff --git a/src/main/java/hudson/plugins/git/util/GitUtils.java b/src/main/java/hudson/plugins/git/util/GitUtils.java index 3d41ad3..68669bb 100644 --- a/src/main/java/hudson/plugins/git/util/GitUtils.java +++ b/src/main/java/hudson/plugins/git/util/GitUtils.java @@ -254,7 +254,8 @@ public class GitUtils implements Serializable { } // add env contributing actions' values from last build to environment - fixes JENKINS-22009 - addEnvironmentContributingActionsValues(env, b); + if (reuseLastBuildEnv) + addEnvironmentContributingActionsValues(env, b); EnvVars.resolve(env);

          Mark Waite added a comment - - edited

          I'm reopening this because Rob's question seems very reasonable, and there is a hint that JENKINS-24516 may be the same issue, manifest in a different manner.

          If it is the same issue, then the challenge becomes writing an automated test which will show the failure. Pull requests of an automated test to show the failure are certainly welcomed...

          Mark Waite added a comment - - edited I'm reopening this because Rob's question seems very reasonable, and there is a hint that JENKINS-24516 may be the same issue, manifest in a different manner. If it is the same issue, then the challenge becomes writing an automated test which will show the failure. Pull requests of an automated test to show the failure are certainly welcomed...

          Rob Langley added a comment -

          Having spent a little more time looking at this, the above will cause JENKINS-22009 to reoccur, I think a better solution would be the following:

          diff --git a/src/main/java/hudson/plugins/git/util/GitUtils.java b/src/main/java/hudson/plugins/git/util/GitUtils.java
index 3d41ad3..d770209 100644
--- a/src/main/java/hudson/plugins/git/util/GitUtils.java
+++ b/src/main/java/hudson/plugins/git/util/GitUtils.java
@@ -266,8 +266,8 @@ public class GitUtils implements Serializable {
         if (buildActions != null) {
             for (Action action : buildActions) {
                 // most importantly, ParametersAction will be processed here (for parameterized builds)
-                if (action instanceof EnvironmentContributingAction) {
-                    EnvironmentContributingAction envAction = (EnvironmentContributingAction) action;
+                if (action instanceof ParametersAction) {
+                    ParametersAction envAction = (ParametersAction) action;
                     envAction.buildEnvVars(b, env);
                 }
             }

          EnvironmentContributingAction is too wider net where are I believe ParametersAction will be much better. At least with my testing it looks to do the right thing with or without EnvInject installe. @danielbeck - it should only add parameters rather then environment variables which should avoid any instance of copying of node specific environments. It also passes the test case added for JENKINS-22009.

          As for a test case, I'm not sure. EnvInject hit this as EnvInjectPluginAction implements EnvironmentContributingAction, is it possible to set something in a test to create some thing similar?

          Rob Langley added a comment - Having spent a little more time looking at this, the above will cause JENKINS-22009 to reoccur, I think a better solution would be the following: diff --git a/src/main/java/hudson/plugins/git/util/GitUtils.java b/src/main/java/hudson/plugins/git/util/GitUtils.java index 3d41ad3..d770209 100644 --- a/src/main/java/hudson/plugins/git/util/GitUtils.java +++ b/src/main/java/hudson/plugins/git/util/GitUtils.java @@ -266,8 +266,8 @@ public class GitUtils implements Serializable { if (buildActions != null) { for (Action action : buildActions) { // most importantly, ParametersAction will be processed here (for parameterized builds) - if (action instanceof EnvironmentContributingAction) { - EnvironmentContributingAction envAction = (EnvironmentContributingAction) action; + if (action instanceof ParametersAction) { + ParametersAction envAction = (ParametersAction) action; envAction.buildEnvVars(b, env); } } EnvironmentContributingAction is too wider net where are I believe ParametersAction will be much better. At least with my testing it looks to do the right thing with or without EnvInject installe. @danielbeck - it should only add parameters rather then environment variables which should avoid any instance of copying of node specific environments. It also passes the test case added for JENKINS-22009 . As for a test case, I'm not sure. EnvInject hit this as EnvInjectPluginAction implements EnvironmentContributingAction, is it possible to set something in a test to create some thing similar?

          Rob Langley added a comment -

          Mark, I just created a PR for this complete with a test case that should catch future regressions: https://github.com/jenkinsci/git-plugin/pull/253

          Thanks,

          Rob Langley added a comment - Mark, I just created a PR for this complete with a test case that should catch future regressions: https://github.com/jenkinsci/git-plugin/pull/253 Thanks,

          Mark Waite added a comment -

          The fix has been merged to the 2.2.x branch and the master branch. It should be first available in the git plugin 2.2.6 release.

          Mark Waite added a comment - The fix has been merged to the 2.2.x branch and the master branch. It should be first available in the git plugin 2.2.6 release.

          Mark Waite added a comment -

          Fixed in git plugin 2.2.6, released 20 Sep 2014

          Mark Waite added a comment - Fixed in git plugin 2.2.6, released 20 Sep 2014

          Mark Waite added a comment -

          Refer to JENKINS-24786 for a possible unintended side effect of fixing this problem.

          Mark Waite added a comment - Refer to JENKINS-24786 for a possible unintended side effect of fixing this problem.

            markewaite Mark Waite
            langers2k Rob Langley
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: