Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38211

JenkinsFiles cannot run on Linux agent of Windows master

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Major Major
    • pipeline

      In a multi-branch pipeline using a JenkinsFile in the SCM it does not appear to be possible to execute 'sh' commands on a Linux node/agent. As per the example below, when using the 'sh' command in the JenkinsFile it causes a 'bad interpreter' error.

      JenkinsFile Snippit:

      node('linux-agent-1') {
          String jdktool = tool name: "1.7", type: 'hudson.model.JDK'
          withEnv(["PATH+MVN=${tool 'Maven 2'}/bin", "PATH+JDK=${jdktool}/bin", "JAVA_HOME=${jdktool}"]) {
              stage 'Checkout'
              checkout scm
      
              stage 'Build'
              try {
                  sh "mvn test"
              } catch (Exception e) {
                  //Notification here!
              }
      }
      

      Error from console log:

      [Pipeline] node
      Running on linux-agent-1 in /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/deveoper-branch-1
      [Pipeline] {
      [Pipeline] stage (Build)
      Entering stage Build
      Proceeding
      [Pipeline] sh
      [developer-branch-1] Running shell script
      sh: /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/developer-branch-1@tmp/durable-6f9c6a38/script.sh: C:\Program: bad interpreter: No such file or directory
      [Pipeline] }
      

          [JENKINS-38211] JenkinsFiles cannot run on Linux agent of Windows master

          Justen Britain added a comment - - edited

          Some additional information that I didn't feel was appropriate for the description as it is a bit speculative. Looking into the "bad interpreter: No such file or directory" output, it seems that most of the time this is related to windows style EOL characters in the script that is attempting to run. I verified that the JenkinsFile did not have the windows specific ^M line endings, to no avail. My hunch as to what is happening is that the command, "sh mvn test" is getting converted to a script, due to this line of output "sh: /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/developer-branch-1@tmp/durable-6f9c6a38/script.sh:" and that script is either:

          1. Getting created on the windows machine and copied to the node causing it to have windows style line endings
          2. Getting created by the windows master on the node causing the windows line endings

          Again this is speculative but seemed worth adding to the comments.

          Justen Britain added a comment - - edited Some additional information that I didn't feel was appropriate for the description as it is a bit speculative. Looking into the "bad interpreter: No such file or directory" output, it seems that most of the time this is related to windows style EOL characters in the script that is attempting to run. I verified that the JenkinsFile did not have the windows specific ^M line endings, to no avail. My hunch as to what is happening is that the command, "sh mvn test" is getting converted to a script, due to this line of output "sh: /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/developer-branch-1@tmp/durable-6f9c6a38/script.sh:" and that script is either: 1. Getting created on the windows machine and copied to the node causing it to have windows style line endings 2. Getting created by the windows master on the node causing the windows line endings Again this is speculative but seemed worth adding to the comments.

          Mark R added a comment -

          Any progress on this? This is a showstopper for us as we are unable to run linux shell scripts.

          Mark R added a comment - Any progress on this? This is a showstopper for us as we are unable to run linux shell scripts.

          Mark R added a comment - - edited

          A coworker was able to grab the temporary script being run on the Linux agent. The line endings are fine (LF-only):

          [agent].../durable-3ca81cbf:
          $ od -cxv script.sh
          0000000   #   !   s   h       -   x   e  \n   e   n   v
                     2123    6873    2d20    6578    650a    766e
          0000014
          

          But he still sees the same error:

          [agent].../durable-3ca81cbf:
          $ ./script.sh
          bash: ./script.sh: sh: bad interpreter: No such file or directory
          

          It looks to be a malformed shebang. Modifying the shebang to be an absolute path allowed the script to run successfully.

          [agent].../durable-3ca81cbf:
          $ od -cxv mod.sh
          0000000   #   !   /   b   i   n   /   s   h       -   x   e  \n   e   n
                     2123    622f    6e69    732f    2068    782d    0a65    6e65
          0000020   v
                     0076
          0000021
          

          I'm now looking into if that is configurable.

          Mark R added a comment - - edited A coworker was able to grab the temporary script being run on the Linux agent. The line endings are fine (LF-only): [agent].../durable-3ca81cbf: $ od -cxv script.sh 0000000 # ! s h - x e \n e n v 2123 6873 2d20 6578 650a 766e 0000014 But he still sees the same error: [agent].../durable-3ca81cbf: $ ./script.sh bash: ./script.sh: sh: bad interpreter: No such file or directory It looks to be a malformed shebang. Modifying the shebang to be an absolute path allowed the script to run successfully. [agent].../durable-3ca81cbf: $ od -cxv mod.sh 0000000 # ! / b i n / s h - x e \n e n 2123 622f 6e69 732f 2068 782d 0a65 6e65 0000020 v 0076 0000021 I'm now looking into if that is configurable.

          Mark R added a comment -

          It comes from this:

          https://github.com/jenkinsci/durable-task-plugin/blob/master/src/main/java/org/jenkinsci/plugins/durabletask/BourneShellScript.java

          String s = script;
                  final Jenkins jenkins = Jenkins.getInstance();
                  if (!s.startsWith("#!") && jenkins != null) {
                      String defaultShell = jenkins.getInjector().getInstance(Shell.DescriptorImpl.class).getShellOrDefault(ws.getChannel());
                      s = "#!"+defaultShell+" -xe\n" + s;
                  }
                  shf.write(s, "UTF-8");
                  shf.chmod(0755);
          

          The default shell for the node must be 'sh'. It should be configurable somewhere... I'll ask our build guys if that can be done. For some reason it doesn't like '#!sh'.

          In the mean time I am able to get it to work by manually setting the shell:

          sh '#!/usr/bin/sh -xe\n env'
          

          Mark R added a comment - It comes from this: https://github.com/jenkinsci/durable-task-plugin/blob/master/src/main/java/org/jenkinsci/plugins/durabletask/BourneShellScript.java String s = script; final Jenkins jenkins = Jenkins.getInstance(); if (!s.startsWith( "#!" ) && jenkins != null ) { String defaultShell = jenkins.getInjector().getInstance(Shell.DescriptorImpl.class).getShellOrDefault(ws.getChannel()); s = "#!" +defaultShell+ " -xe\n" + s; } shf.write(s, "UTF-8" ); shf.chmod(0755); The default shell for the node must be 'sh'. It should be configurable somewhere... I'll ask our build guys if that can be done. For some reason it doesn't like '#!sh'. In the mean time I am able to get it to work by manually setting the shell: sh '#!/usr/bin/sh -xe\n env'

          Liam Newman added a comment -

          I discussed this with bobthemagicman at CD Summit yesterday.

          This is at least partly being caused by a strange configuration on that specific Windows-based Jenkins Master. The "Shell Executable" has been manually set to "C:\Program Files (x86)\Git\git-bash.exe". This means that the sh step on all nodes (master and agents) will add #! C:\Program Files (x86)\Git\git-bash.exe -xe to the front of their shell scripts. On windows, I believe this gets ignored, but on linux it blows up.

          If we treat this as a feature request it would be that there should be a "Shell Executable" configuration available on each node agent that inherits from global configuration by default. Or treating "shell" as a tool that one can configure (like ant, maven, JDK, etc).

          But I'd like let's talk about work-arounds that bobthemagicman can use (aside from switching to a linux master):
          1. Set the "Shell executable" configuration value to empty. That is what is by default, and generally is correct. Then make sure you have Windows Git installed, and add C:\Program Files\Git\bin to the path on all windows machines that run Jenkins agents (including master).
          (based on http://stackoverflow.com/a/35045651/1637252)

          2. If you cannot change the shell executable setting, you can get around this by manually adding #! to the start of all your sh steps, but this really is a distant second.

          The third point is that this and other difficulties and edge cases around running jenkins (either master or agents) need to be documented. I'll get a page started on jenkins.io and discuss this.

          Liam Newman added a comment - I discussed this with bobthemagicman at CD Summit yesterday. This is at least partly being caused by a strange configuration on that specific Windows-based Jenkins Master. The "Shell Executable" has been manually set to "C:\Program Files (x86)\Git\git-bash.exe". This means that the sh step on all nodes (master and agents) will add #! C:\Program Files (x86)\Git\git-bash.exe -xe to the front of their shell scripts. On windows, I believe this gets ignored, but on linux it blows up. If we treat this as a feature request it would be that there should be a "Shell Executable" configuration available on each node agent that inherits from global configuration by default. Or treating "shell" as a tool that one can configure (like ant, maven, JDK, etc). But I'd like let's talk about work-arounds that bobthemagicman can use (aside from switching to a linux master): 1. Set the "Shell executable" configuration value to empty. That is what is by default, and generally is correct. Then make sure you have Windows Git installed, and add C:\Program Files\Git\bin to the path on all windows machines that run Jenkins agents (including master). (based on http://stackoverflow.com/a/35045651/1637252 ) 2. If you cannot change the shell executable setting, you can get around this by manually adding #! to the start of all your sh steps, but this really is a distant second. The third point is that this and other difficulties and edge cases around running jenkins (either master or agents) need to be documented. I'll get a page started on jenkins.io and discuss this.

          Liam Newman added a comment - - edited

          Liam Newman added a comment - - edited This is where the default gets provided: https://github.com/jenkinsci/jenkins/blob/720b75f096e573b102323b529ec952e2106d4a57/core/src/main/java/hudson/tasks/Shell.java#L155

          Andrew Bayer added a comment -

          Is this still relevant?

          Andrew Bayer added a comment - Is this still relevant?

            jglick Jesse Glick
            bobthemagicman Justen Britain
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: