Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44930

Allow sh to return exit status, stdout and stderr all at once

      Like many of the commenters on -JENKINS-26133- I'd like to be able to capture the exit status and text written to standard out at the same time.

      My current use case is calling git merge --no-edit $branches and if there was an error sending a slack notification with the output. 

      The current workaround is:

      def status = sh(returnStatus: true, script: "git merge --no-edit $branches > merge_output.txt")
      if (status != 0) {
        currentBuild.result = 'FAILED'
        def output = readFile('merge_output.txt').trim()
        slackSend channel: SLACK_CHANNEL, message: "<${env.JOB_URL}|${env.JOB_NAME}> ran into an error merging the PR branches into the ${TARGET_BRANCH} branch:\n```\n${output}\n```\n<${env.BUILD_URL}/console|See the full output>", color: 'warning', tokenCredentialId: 'slack-token'
        error 'Merge conflict'
      }
      sh 'rm merge_output.txt'

      Which works but isn't a great developer experience... It would be great if I could request an object that contained: status, stdout, and stderr.

          [JENKINS-44930] Allow sh to return exit status, stdout and stderr all at once

          Levi Blaney added a comment -

          I also have wanted to be able to have the exitCode and the stdOut at the same time. About a year ago I started what I call the Jenkins Standard Library to help make building pipelines faster and easier. One of the first things I solved was these issues with `sh()`.

          @Library('jenkins-std-lib')
          import org.dsty.bash.BashClient
          import org.dsty.bash.ScriptError
          
          node() {    
          
              String msg = 'TestMessage'
          
              def bash = new BashClient(this)
          
              def result = bash("echo '${msg}'")
          
              if (result.stdOut != msg ) {
                  error('Did not contain correct output.')
              }
          
              if ( result.stdErr ) {
                  error('Should not have output anything.')
              }    
          
              if ( result.exitCode != 0 ) {
                  error('Exited with wrong code.')
              }    
          
              def exception = null
          
              try {
                  bash('fakecommand')
              } catch (ScriptError e) {
                  exception = e
              }    
          
              if ( !exception.stdErr.contains('fakecommand: command not found') ) {
                  error('Command was found.')
              }   
            
              if (exception.stdOut) {
                  error('Should not have stdOut.')
              }    
          
              if ( exception.exitCode != 127) {
                  error('Exited with wrong code.')
              }
          }
          
          

          Also not documented here is `result.output` which contains both the stdErr and stdOut but the order can't be guaranteed because bash is async. I also made the exception the same as the result because why wouldn't you want to include things like stdErr, stdOut and exitCode in your failure notification. Also `bash.silent()` which will not output ANYTHING to the build console. That way you can keep your build logs short and concise. Finally `bash.ignoreErrors()` which will return the result object even if the script errors. It also supports not sending output to the build console. 

          You can also browse the full documentation.

          I stopped working on the project when I couldn't find a way to test the code completely but about 2 weeks ago I found a new experimental way to test my library and now that I'm able to completely test the codebase I'm back to adding more features. 

          Right now the library only has the BashClient and LogClient but I think next I will add an easy way to POST and GET instead of dropping down to curl all the time. I think after that maybe github status checks or maybe a better slack/teams notification.

          It would be awesome if you would use my library and open github issues for features/feedback. If not you can always look at the BashClient code and see how I made it work and use it in your own shared libraries. The code is completely OpenSource and free. I will also work on a contributors guide if anyone is interested. 

          drewish cyberwizzard childnode mpettigr joanperez too many to tag so I will stop here.

          Levi Blaney added a comment - I also have wanted to be able to have the exitCode and the stdOut at the same time. About a year ago I started what I call the Jenkins Standard Library  to help make building pipelines faster and easier. One of the first things I solved was these issues with `sh()`. @Library( 'jenkins-std-lib' ) import org.dsty.bash.BashClient import org.dsty.bash.ScriptError node() { String msg = 'TestMessage' def bash = new BashClient( this ) def result = bash( "echo '${msg}' " ) if (result.stdOut != msg ) { error( 'Did not contain correct output.' ) } if ( result.stdErr ) { error( 'Should not have output anything.' ) } if ( result.exitCode != 0 ) { error( 'Exited with wrong code.' ) } def exception = null try { bash( 'fakecommand' ) } catch (ScriptError e) { exception = e } if ( !exception.stdErr.contains( 'fakecommand: command not found' ) ) { error( 'Command was found.' ) } if (exception.stdOut) { error( 'Should not have stdOut.' ) } if ( exception.exitCode != 127) { error( 'Exited with wrong code.' ) } } Also not documented here is `result.output` which contains both the stdErr and stdOut but the order can't be guaranteed because bash is async. I also made the exception the same as the result because why wouldn't you want to include things like stdErr, stdOut and exitCode in your failure notification. Also `bash.silent()` which will not output ANYTHING to the build console. That way you can keep your build logs short and concise. Finally `bash.ignoreErrors()` which will return the result object even if the script errors. It also supports not sending output to the build console.  You can also browse the full documentation . I stopped working on the project when I couldn't find a way to test the code completely but about 2 weeks ago I found a new experimental way to test my library and now that I'm able to completely test the codebase I'm back to adding more features.  Right now the library only has the BashClient and LogClient but I think next I will add an easy way to POST and GET instead of dropping down to curl all the time. I think after that maybe github status checks or maybe a better slack/teams notification. It would be awesome if you would use my library and open github issues for features/feedback. If not you can always look at the BashClient code and see how I made it work and use it in your own shared libraries. The code is completely OpenSource and free. I will also work on a contributors guide if anyone is interested.  drewish cyberwizzard childnode mpettigr joanperez  too many to tag so I will stop here.

          Wow, open since 2017, amazing. This is not still controversial, right? I believe there's a lot of tech debt that makes this hard, but I can't imagine why this is controversial.

          Dharma Indurthy added a comment - Wow, open since 2017, amazing. This is not still controversial, right? I believe there's a lot of tech debt that makes this hard, but I can't imagine why this is controversial.

          Haralds added a comment - - edited

           

          Hi everyone, I've idea how to improve on this. Here is exact plan:

          https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/2a88fdd885caa0d1bcf3336efc00b579b75ced82/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java#L648

           

          if ((returnStatus && originalCause == null) || exitCode == 0) {  
          

           

          change to something like (in case when both stdout and status code are enabled): 

           

          if ((returnStatus && originalCause == null) || exitCode == 0 || (returnStatus && originalCause == null && returnStdout ) ) {
          

           

          Then in case if both are enabled then we will need extra if  like:

           

          if (returnStdOut && returnStatus) {
            getContext().onSuccess(["status":exitCode, "output": new String(output.produce(), StandardCharsets.UTF_8)]);
          else {
             getContext().onSuccess(returnStatus ? exitCode : returnStdout ? new String(output.produce(), StandardCharsets.UTF_8) : null);
          }  

           

          That would maintain compatibility with previous code. Also would need to update (or remove) https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/2a88fdd885caa0d1bcf3336efc00b579b75ced82/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java#L168 to allow both parameters in same time.

           

          Haralds added a comment - - edited   Hi everyone, I've idea how to improve on this. Here is exact plan: https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/2a88fdd885caa0d1bcf3336efc00b579b75ced82/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java#L648   if ((returnStatus && originalCause == null ) || exitCode == 0) {     change to something like (in case when both stdout and status code are enabled):    if ((returnStatus && originalCause == null ) || exitCode == 0 || (returnStatus && originalCause == null && returnStdout ) ) {   Then in case if both are enabled then we will need extra if  like:   if (returnStdOut && returnStatus) { getContext().onSuccess([ "status" :exitCode, "output" : new String (output.produce(), StandardCharsets.UTF_8)]); else { getContext().onSuccess(returnStatus ? exitCode : returnStdout ? new String (output.produce(), StandardCharsets.UTF_8) : null ); }   That would maintain compatibility with previous code. Also would need to update (or remove)  https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/2a88fdd885caa0d1bcf3336efc00b579b75ced82/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java#L168  to allow both parameters in same time.  

          m added a comment -

          I have a similar issue, except it's with bat calls. Is there a similar ticket for this or should it be handled here too?

          m added a comment - I have a similar issue, except it's with bat calls. Is there a similar ticket for this or should it be handled here too?

          Ian Boudreaux added a comment -

          It would be great if Jenkins supported similar functionality for powershell as well.

          Ian Boudreaux added a comment - It would be great if Jenkins supported similar functionality for powershell as well.

          shadycuz instead of developing a curl wrapper, you can use the HTTP Request plugin. It already returns the HTTP status code and the content in a single call.

          Martin d'Anjou added a comment - shadycuz  instead of developing a curl wrapper, you can use the HTTP Request  plugin. It already returns the HTTP status code and the content in a single call.

          Levi Blaney added a comment - - edited

          deepchip I see a lot of teams struggle with plugins for whatever reason. My library uses a groovy native implementation based on pythons requests library. You can see the docs here https://javadoc.io/doc/io.github.dontshavetheyak/jenkins-std-lib/latest/org/dsty/http/Requests.html

          Levi Blaney added a comment - - edited deepchip I see a lot of teams struggle with plugins for whatever reason. My library uses a groovy native implementation based on pythons requests library. You can see the docs here https://javadoc.io/doc/io.github.dontshavetheyak/jenkins-std-lib/latest/org/dsty/http/Requests.html

          Pawel added a comment -

          Is this still unimplemented ? Really!?

          Pawel added a comment - Is this still unimplemented ? Really!?

          Daniel added a comment - - edited

          +1 for this issue, hope it helps to move it forward. It's open since 2017!

          Daniel added a comment - - edited +1 for this issue, hope it helps to move it forward. It's open since 2017!

          Noam Manos added a comment -

          Noam Manos added a comment - Hi, see my workaround for this 2017 feature https://stackoverflow.com/questions/68967642/how-to-return-stdout-and-stderr-together-with-the-status-from-a-jenkins-pipeline/77900872#77900872 Hope it helps!

            Unassigned Unassigned
            drewish andrew morton
            Votes:
            167 Vote for this issue
            Watchers:
            144 Start watching this issue

              Created:
              Updated: