Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50902

shell pipeline step cannot use relative path to shell

    • durable-task 1.28

      There is some difference between free-style project shell build step and the pipeline `sh` step. If I set in jenkins global configuration `bash` as the "Shell Executable", then free-style project works and calls `bash` from the PATH on the slave. The pipeline `sh` step fails like this though:

      [pipeline-test-job] Running shell script
      sh: /home/jenkins/workspace/pipeline-test-akostadi@tmp/durable-2ff6e6ae/script.sh: bash: bad interpreter: No such file or directory
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: script returned exit code 126
      Finished: FAILURE
      

      The use case is that some systems like MacOS ship with an old bash version under /bin/sh. And we want to not be limited in shell scripts by that version. Since proper bash should be installed in other locations on such slaves, we need to run `bash` from PATH. But also can't hardcode `bash` absolute path because it is still `/bin/bash` on normal linux slaves.

      I'm reading through ShellStep.java and BourneShellScript.java.

      It looks like it launches `sh -c cmd` but I don't see how the global setting is applied, even less I understand why it fails. At the same time free-style projects work fine with this setting on all slaves.

      I think this difference between free-style and pipeline might be related to JENKINS-44341 also.

      Any clue what exactly might be the difference and how it could be fixed?

          [JENKINS-50902] shell pipeline step cannot use relative path to shell

          The problem here seems to be that BourneShellScript is using shebang approach to choose interpreter while ShellStep calls it directly. I have not found a reason why pipeline impl has chosen such approach in 2014 so I am attempting to unify this.

          A number of customers have run into situation of combining freestyle/matrix with pipeline jobs while utilizing windows/linux/unix hosts and we failed to identify a setting what would work here (unless one wants to compromise on subset of shell features). An alternative is to hardcode shebangs in all script invocations and use different absolute paths for windows.

          Oliver Gondža added a comment - The problem here seems to be that BourneShellScript is using shebang approach to choose interpreter while ShellStep calls it directly. I have not found a reason why pipeline impl has chosen such approach in 2014 so I am attempting to unify this. A number of customers have run into situation of combining freestyle/matrix with pipeline jobs while utilizing windows/linux/unix hosts and we failed to identify a setting what would work here (unless one wants to compromise on subset of shell features). An alternative is to hardcode shebangs in all script invocations and use different absolute paths for windows.

          Fix proposed

          Oliver Gondža added a comment - Fix proposed

          Devin Nusbaum added a comment -

          The fix for this was just released in version 1.28 of the Durable Task plugin. Thanks olivergondza!

          Devin Nusbaum added a comment - The fix for this was just released in version 1.28 of the Durable Task plugin. Thanks  olivergondza !

          I am getting the same issue after updating durable-task to 1.28 and restart Jenkins.

          We have a shell script which takes a couple of arguments (the environment, the browser, test type, tag, etc.). The command is as follows:

          sh /root/runtest.sh $DEVICE $ENVIRONMENT $TYPE $IS_GROUP $PRODUCT $TAG
          

          This invokes codeception tool to trigger the following command to run :

           

          /root/test/vendor/bin/codecept run acceptance $PRODUCT\\${TAG}Cest.php --steps --html='index.html'
          

          This works fine when we use a Freestyle Project, but when I started to add the same step to my Pipeline, it cannot identify the arguments, especially the relative path used for testing.

          Let me know if any more info is needed.

           

          Amir O. Khosrowshahi added a comment - I am getting the same issue after updating durable-task to 1.28 and restart Jenkins. We have a shell script which takes a couple of arguments (the environment, the browser, test type, tag, etc.). The command is as follows: sh /root/runtest.sh $DEVICE $ENVIRONMENT $TYPE $IS_GROUP $PRODUCT $TAG This invokes codeception tool to trigger the following command to run :   /root/test/vendor/bin/codecept run acceptance $PRODUCT\\${TAG}Cest.php --steps --html= 'index.html' This works fine when we use a Freestyle Project, but when I started to add the same step to my Pipeline, it cannot identify the arguments, especially the relative path used for testing. Let me know if any more info is needed.  

          Oliver Gondža added a comment - - edited

          aluvian, your issue appears unrelated. This was never intended to change the argument resolution (that is supposed to be relative to pwd of the pipeline) but the hardwired interpreter resolution.

          Oliver Gondža added a comment - - edited aluvian , your issue appears unrelated. This was never intended to change the argument resolution (that is supposed to be relative to pwd of the pipeline) but the hardwired interpreter resolution.

            olivergondza Oliver Gondža
            akostadinov akostadinov
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: