• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Critical Critical
    • blueocean-plugin, pipeline
    • None
    • Jenkins LTS v2.46.3
      Pipeline plugin v2.5
      Blue Ocean v1.1.4
    • Pipeline: Basic Steps 2.16

      Problem
      There is no way to catch a failing block, mark it as "failed" with a user defined description and continue execution of the Pipeline.

      In the example below, the Optional Tests stage would always be successful.

      stage ('Optional Tests') {
              try {
                  echo "Running optional tests..."
                  sh 'exit -1'
              } catch (Exception err) {
                  echo 'Optional tests failed... don't propagate failure'
              }
      }
      

      Solution
      Introduce new steps that can set the state of the stage to UNSTABLE, FAILED, SUCCESS, ABORTED. The message specified would be visible on the UI in Blue Ocean.

      Example

      stage ('Optional Tests') {
              try {
                  echo "Running optional tests..."
                  sh 'exit -1'
              } catch (Exception err) {
                  unstable 'Optional tests failed... don't propagate failure'
              }
      }
      

      Notes

      • There is precedence here with the error signal step.
      • To ensure consistency the FAILED state should always fail the pipeline.
      • The UNSTABLE state is used by tests to mark a stage as problematic but allows execution of the Pipeline to continue.
      • Unstable should be expanded to handle the case where the intent of the Pipeline author is to record that there was a problem with the execution of the stage but execution is allowed to continue.

      Original Request
      I recently discovered that pipeline stages that are run in parallel which report different results from one another are not currently being displayed correctly in the Blue Ocean UI. Here is a short snippet of code that reproduces the problem case I found:

      node {
          parallel testA: {
              stage ("Required Tests") {
                  echo "Running required tests..."
              }
          }, testB: {
              try {
                  stage ("Optional Tests") {
                      echo "Running optional tests..."
                      sh 'exit -1'
                  }
              } catch (Exception err) {
                  echo "Optional tests failed... don't propagate failure"
              }
          }
          
          stage ("Deploy") {
              echo "Deploying..."
          }
      }
      

      Here I expect that 2 dummy test suites will be executed, one containing important tests that must all pass, and another containing less important tests that may be allowed to fail. Following these 2 parallel test stages is a final deploy stage that would be run afterwards so long as the "important" tests complete successfully.

      In this example I'd expect that the node in the pipeline graph for the 'testB' stage would show up red while the other 2 nodes would show up green. Currently this does not happen - all 3 stages show up as green. I've attached a sample screenshot to illustrate the results.

      It is also worth mentioning that the statuses of these stages does appear to be represented correctly in the old 'stage view' from the default Jenkins UI. I've attached a screenshot illustrating this as well.

          [JENKINS-45579] Step to set stage or parallel status

          it may be worth noting that, when not using a parallel code block in my example above the Blue Ocean UI does correctly render the nodes in the appropriate color.

          Kevin Phillips added a comment - it may be worth noting that, when not using a parallel code block in my example above the Blue Ocean UI does correctly render the nodes in the appropriate color.

          James Dumay added a comment -

          Hi leedega,

          Thanks for the report. I really appreciate the clear example demonstrating this.

          Unfortunately, by catching the exception and eching, Blue Ocean has correctly interpreted the status of this stage as you have effectively eaten the error in the catch block. Not possible for us to determine the exact intent of your program automatically.

          You'll have to find a different way to do this that causes the parallel to fail and bubble up this info. Commonly people set currentBuild.result = "UNSTABLE" when skipping over test failures. At the moment, that marks the whole pipeline as unstable but we are currently working on a way to set unstable for a single step/stage/parallel.

          Thanks
          James

          James Dumay added a comment - Hi leedega , Thanks for the report. I really appreciate the clear example demonstrating this. Unfortunately, by catching the exception and eching, Blue Ocean has correctly interpreted the status of this stage as you have effectively eaten the error in the catch block. Not possible for us to determine the exact intent of your program automatically. You'll have to find a different way to do this that causes the parallel to fail and bubble up this info. Commonly people set currentBuild.result = "UNSTABLE" when skipping over test failures. At the moment, that marks the whole pipeline as unstable but we are currently working on a way to set unstable for a single step/stage/parallel. Thanks James

          James Dumay added a comment -

          leedega I am discussing with our team internally how we might be able to solve this problem but in a slightly different way. Requires the issues that you already linked here.

          James Dumay added a comment - leedega I am discussing with our team internally how we might be able to solve this problem but in a slightly different way. Requires the issues that you already linked here.

          James Dumay added a comment -

          leedega I've updated the description of this ticket to reflect a proposed solution. Please let me know if this looks good to you.

          James Dumay added a comment - leedega I've updated the description of this ticket to reflect a proposed solution. Please let me know if this looks good to you.

          jamesdumay

          Can you also please have that status extractable externally i.e. in a mailing groovy script. Usually the groovy scripts would use build result to show specific parts depending only on currentBuild.result causes problems when multiple parallel tasks tries to set the status just for the e-mail then return the status back to the original state.

           

           

          mohamed badran added a comment - jamesdumay Can you also please have that status extractable externally i.e. in a mailing groovy script. Usually the groovy scripts would use build result to show specific parts depending only on currentBuild.result causes problems when multiple parallel tasks tries to set the status just for the e-mail then return the status back to the original state.    

          Nick Jones added a comment -

          Given that JENKINS-27092 has been closed with the intent to cover its needs on this one, I just wanted to add my need here: a simple way to abort a build while still reporting a successful status check back to GitHub (in a multibranch pipeline scenario). Right now I'm using a when condition on every stage to skip it (having detected a message like "[ci skip]" in the git commit log).

          Nick Jones added a comment - Given that JENKINS-27092 has been closed with the intent to cover its needs on this one, I just wanted to add my need here: a simple way to abort a build while still reporting a successful status check back to GitHub (in a multibranch pipeline scenario). Right now I'm using a when condition on every stage to skip it (having detected a message like " [ci skip] " in the git commit log).

          I'm having similar issues due to the unreadability of the pipeline, the Devs are complaining about it. Do we have any news on this feature of setting parallel steps individual results?

          Felyppe Rodrigues added a comment - I'm having similar issues due to the unreadability of the pipeline, the Devs are complaining about it. Do we have any news on this feature of setting parallel steps individual results?

          Jacob Keller added a comment -

          This is also somewhat related to steps like the warnings publisher, which if you run in parallel results in the whole build being marked unstable even though only a single segment should have been, which may not be fixed by making separate unstable results for each parallel step (unless it fully encapsulates the build?)

          Jacob Keller added a comment - This is also somewhat related to steps like the warnings publisher, which if you run in parallel results in the whole build being marked unstable even though only a single segment should have been, which may not be fixed by making separate unstable results for each parallel step (unless it fully encapsulates the build?)

          Pavel Rogovoy added a comment -

          Our Dev guys are also complaining about this issue. We would like to be able to mark certain stages as unstable / failed and still continue to execute the pipeline. Then we need the ability to check for any failures of the current run and not to execute the deploy stage. What do you think?

          Pavel Rogovoy added a comment - Our Dev guys are also complaining about this issue. We would like to be able to mark certain stages as unstable / failed and still continue to execute the pipeline. Then we need the ability to check for any failures of the current run and not to execute the deploy stage. What do you think?

          UP!  I'd definitely be very happy to see that workint too though

          Tanguy von Stebut added a comment - UP!  I'd definitely be very happy to see that workint too though

          Kasia Gauza added a comment -

          What's the status of the task?

          It's strange that after a year+ of trying to switch our CI from Build Flow to Pipelines/Blue Ocean, I still find the the former more usable...

          Kasia Gauza added a comment - What's the status of the task? It's strange that after a year+ of trying to switch our CI from Build Flow to Pipelines/Blue Ocean , I still find the the former more usable...

          Elie Kassis added a comment -

          Is there any progress on this issue?

          Elie Kassis added a comment - Is there any progress on this issue?

          HI, is there any update ? We really need that feature

          Sergey Izgiyaev added a comment - HI, is there any update ? We really need that feature

          Hi ! We really need this ! No news ?

          Paul de Courcel added a comment - Hi ! We really need this ! No news ?

          Bilel Yahia added a comment -

          I'd love to see this implemented too. Is there any planned work to do it?

          Bilel Yahia added a comment - I'd love to see this implemented too. Is there any planned work to do it?

          Shani Dar added a comment -

          Is there any update? 

          Shani Dar added a comment - Is there any update? 

          Ian Driver added a comment - - edited

          We also need this feature

          Ian Driver added a comment - - edited We also need this feature

          Jesse Glick added a comment -
          stage ('Optional Tests') {
            echo 'Running optional tests…'
            if (sh(script: 'exit -1', returnStatus: true) != 0) {
              unstable 'Optional tests failed…do not propagate failure'
            }
          }
          

          Jesse Glick added a comment - stage ( 'Optional Tests' ) { echo 'Running optional tests…' if (sh(script: 'exit -1' , returnStatus: true ) != 0) { unstable 'Optional tests failed… do not propagate failure' } }

          Stuart Rowe added a comment - - edited

          jglick is unstable a step now? I thought it was only a condition for post in declarative pipelines.

          Stuart Rowe added a comment - - edited jglick is unstable a step now? I thought it was only a condition for post in declarative pipelines.

          Jesse Glick added a comment -

          No, sorry for confusion. I was just sketching an edit to the original example by leedega to not rely on a try-catch block, which would be undesirable in this context as it would catch not just nonzero exit status from a script but all sorts of other errors and even build interrupts and timeouts.

          Jesse Glick added a comment - No, sorry for confusion. I was just sketching an edit to the original example by leedega to not rely on a try - catch block, which would be undesirable in this context as it would catch not just nonzero exit status from a script but all sorts of other errors and even build interrupts and timeouts.

          Saad Azam added a comment -

          Another thing is that the behavior described earlier about the fact that build stage shows the correct node rendering when not using parallel (link to the comment below) does not hold true for a declarative pipeline.

          https://issues.jenkins-ci.org/browse/JENKINS-45579?focusedCommentId=307227&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-307227

          Saad Azam added a comment - Another thing is that the behavior described earlier about the fact that build stage shows the correct node rendering when not using parallel (link to the comment below) does not hold true for a declarative pipeline. https://issues.jenkins-ci.org/browse/JENKINS-45579?focusedCommentId=307227&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-307227

          Nofar Bluestein added a comment - - edited

          Hello, 

          I'm a Product Manager at CloudBees and we're currently working on a design for this request, and I would love to get your feedback on the suggested solution:

          We were thinking of expanding the functionality of "catchError]" block as follows:

          Current default behavior (will be maintained for backward compatibility) when an error is caught:

          • Step result is failed (though execution of the pipeline continues)
          • Stage result is success
          • Build result is failed

          Suggested changes:

          Add ability to set arguments to determine the desired results upon a caught error:

          • Stage result : success/failure (default: success)
          • Build result: success/failure (default: failure)

           

          In addition, a very common use case is the ability to set a warning - mark the build and a stage in it as one that warrants attention without failing the build. To support that, we're considering adding a new block called "warnError", which will behave as follows:

          • If an error is caught:
            • Step result is failed
            • Stage result is set to unstable
            • Build result is set to unstable

          (we're also fixing the issue where if a build becomes unstable it's hard to pinpoint which stage caused the build to become unstable )

           

          Thank you, 

          Nofar Bluestein,

          Product Manager, CloudBees

           

          Nofar Bluestein added a comment - - edited Hello,  I'm a Product Manager at CloudBees and we're currently working on a design for this request, and I would love to get your feedback on the suggested solution: We were thinking of expanding the functionality of " catchError ]" block as follows: Current default behavior (will be maintained for backward compatibility) when an error is caught: Step result is failed (though execution of the pipeline continues) Stage result is success Build result is failed Suggested changes: Add ability to set arguments to determine the desired results upon a caught error: Stage result : success/failure (default: success) Build result: success/failure (default: failure)   In addition, a very common use case is the ability to set a warning - mark the build and a stage in it as one that warrants attention without failing the build. To support that, we're considering adding a new block called " warnError ", which will behave as follows: If an error is caught: Step result is failed Stage result is set to unstable Build result is set to unstable (we're also fixing the issue where if a build becomes unstable it's hard to pinpoint which stage caused the build to become unstable )   Thank you,  Nofar Bluestein, Product Manager, CloudBees  

          nofarblue - Sounds great. I'm not too concerned about syntax decisions, as long as the basic issues are addressed. This sounds like a very reasonable solution. Glad the issue is getting some attention!

          Brian Villanueva added a comment - nofarblue - Sounds great. I'm not too concerned about syntax decisions, as long as the basic issues are addressed. This sounds like a very reasonable solution. Glad the issue is getting some attention!

          Saad Azam added a comment -

          nofarblue - Many thanks for the response. The changes you suggested are exactly what I am looking forward to have in declarative pipeline. In case it is not too much of an ask for you, it would be nice to get a rough estimate about when are the suggested changes going to be rolled out. 

          Saad Azam added a comment - nofarblue - Many thanks for the response. The changes you suggested are exactly what I am looking forward to have in declarative pipeline. In case it is not too much of an ask for you, it would be nice to get a rough estimate about when are the suggested changes going to be rolled out. 

          nofarblue - Many thanks for looking into this.  Your proposal sounds good.  I am particularly interested in the handling of the unstable condition.  Our pipelines are mostly scripted, and we have been using the error() function to mark failed stages, but it would help to be able to mark stages as unstable and keep going.  Does your current plan include the addition of an unstable() step?

          Nancy Robertson added a comment - nofarblue - Many thanks for looking into this.  Your proposal sounds good.  I am particularly interested in the handling of the unstable condition.  Our pipelines are mostly scripted, and we have been using the error() function to mark failed stages, but it would help to be able to mark stages as unstable and keep going.  Does your current plan include the addition of an unstable() step?

          Nofar Bluestein added a comment - - edited

          Thank you for the feedback!

          saad_azam I cannot commit to an ETA at the moment, but I can tell you it's a high priority for us,. I will provide additional update soon. 

          nancyrobertson2005 in scripted pipelines the following should work: 

           `warnError { error() }` would have the same result, but I will consider adding 'unstable' step so it's more elegant and consistent with the 'error' step

           

          Nofar Bluestein added a comment - - edited Thank you for the feedback! saad_azam I cannot commit to an ETA at the moment, but I can tell you it's a high priority for us,. I will provide additional update soon.  nancyrobertson2005 in scripted pipelines the following should work:   `warnError { error() }` would have the same result, but I will consider adding 'unstable' step so it's more elegant and consistent with the 'error' step  

          Shani Dar added a comment -

          nofarblue Thanks for the update! looking forward to use the stage status solution.  

          Shani Dar added a comment - nofarblue  Thanks for the update! looking forward to use the stage status solution.  

          Devin Nusbaum added a comment -

          Pipeline: Basic Steps Plugin version 2.16 added the new warnError and unstable steps and changes to catchError that nofarblue mentioned, see the 2.16 release notes for details on the specifics. See this comment on JENKINS-39203 for additional information on related issues. You need to update to Pipeline: Graph Analysis 1.10 for these steps to be visualized correctly by Blue Ocean.

          Should we close this issue and open more specific issues related to the request for a step to set the build result to FAILURE but continue execution and a step to set the build result to ABORTED and abort the build, or handle them as part of this ticket?

          Devin Nusbaum added a comment - Pipeline: Basic Steps Plugin version 2.16 added the new warnError and unstable steps and changes to catchError that nofarblue mentioned, see the 2.16 release notes for details on the specifics. See this comment on JENKINS-39203 for additional information on related issues. You need to update to Pipeline: Graph Analysis 1.10 for these steps to be visualized correctly by Blue Ocean. Should we close this issue and open more specific issues related to the request for a step to set the build result to FAILURE but continue execution and a step to set the build result to ABORTED and abort the build, or handle them as part of this ticket?

          RG added a comment -

          Hi. Would you mind adding an example in https://github.com/jenkinsci/pipeline-examples ? I'm not exactly an expert and coming up with the right syntax  assembling bit of code & release note is quite hard.

          RG added a comment - Hi. Would you mind adding an example in https://github.com/jenkinsci/pipeline-examples ? I'm not exactly an expert and coming up with the right syntax  assembling bit of code & release note is quite hard.

          Saad Azam added a comment -

          dnusbaum I have tried the latest update with newly added 'unstable' step as well as updated 'catchError'. I am now able to setup and visualize individual stage status independently. Many thanks.

          Saad Azam added a comment - dnusbaum  I have tried the latest update with newly added 'unstable' step as well as updated 'catchError'. I am now able to setup and visualize individual stage status independently. Many thanks.

          Devin Nusbaum added a comment -

          rrrrrrrr You can use the "Pipeline Syntax" link in the sidebar of a Pipeline job in Jenkins to get a visual interface that will show you the options provided by catchError, warnError, and unstable and will allow you to generate the actual syntax that would go in a Pipeline given the options you specify.

          I will look into adding something to the examples repo.

          Devin Nusbaum added a comment - rrrrrrrr You can use the "Pipeline Syntax" link in the sidebar of a Pipeline job in Jenkins to get a visual interface that will show you the options provided by catchError , warnError , and unstable and will allow you to generate the actual syntax that would go in a Pipeline given the options you specify. I will look into adding something to the examples repo.

          Brian J Murrell added a comment - - edited

          I used the Pipeline Syntax snippet generator to generate me a catchError step and it produced:

          catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
           // some block
          }
          

          which then chokes Jenkins with:

          WorkflowScript: 147: Expecting "class hudson.model.Result" for parameter "buildResult" but got "SUCCESS" of type class java.lang.String instead @ line 147, column 49.
                   catchError(buildResult: 'SUCCESS',
                                           ^
          

          Removing the single quotes surrounding the SUCCESS and FAILURE values seems to at least keep the linter happy and the pipeline at least runs. Still waiting to see if these new values have the desired effect though.

          Brian J Murrell added a comment - - edited I used the Pipeline Syntax snippet generator to generate me a catchError step and it produced: catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { // some block } which then chokes Jenkins with: WorkflowScript: 147: Expecting "class hudson.model.Result" for parameter "buildResult" but got "SUCCESS" of type class java.lang.String instead @ line 147, column 49. catchError(buildResult: 'SUCCESS', ^ Removing the single quotes surrounding the SUCCESS and FAILURE values seems to at least keep the linter happy and the pipeline at least runs. Still waiting to see if these new values have the desired effect though.

          Devin Nusbaum added a comment -

          brianjmurrell Yes, turns out that Declarative doesn't like the parameter types for `catchError`. See JENKINS-57537 for more information, I am working on a fix that should be ready soon.

          Devin Nusbaum added a comment - brianjmurrell Yes, turns out that Declarative doesn't like the parameter types for `catchError`. See JENKINS-57537 for more information, I am working on a fix that should be ready soon.

          dnusbaum Oh, this is a pity.    catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') was going to solve exactly the problem I newly have on hand today, which is how to let a step fail, mark it's stage as failed but not fail the overall job.

          I'm really loathed to make the step and stage look like it succeeded (because it will mask failures that need investigating) just so that the rest of the stages get run and overall job passes.

          Brian J Murrell added a comment - dnusbaum Oh, this is a pity.     catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')  was going to solve exactly the problem I newly have on hand today, which is how to let a step fail, mark it's stage as failed but not fail the overall job. I'm really loathed to make the step and stage look like it succeeded (because it will mask failures that need investigating) just so that the rest of the stages get run and overall job passes.

          dnusbaum New issue filed about catchError(), and the post processing block, FWIW.

          Brian J Murrell added a comment - dnusbaum New issue filed about catchError() , and the post processing block, FWIW.

          Devin Nusbaum added a comment -

          The original example in the description of this ticket was fixed as part of JENKINS-39203 with the new warnError and unstable steps and changes to the catchError step, so I am going to close this issue See https://jenkins.io/blog/2019/07/05/jenkins-pipeline-stage-result-visualization-improvements/ and this earlier comment for details. if you have a request for a step that is related to the general title of the ticket but is not covered by the changes I mentioned, please file a new ticket along the lines of JENKINS-27092.

          Devin Nusbaum added a comment - The original example in the description of this ticket was fixed as part of JENKINS-39203 with the new warnError  and unstable steps and changes to the catchError step, so I am going to close this issue See https://jenkins.io/blog/2019/07/05/jenkins-pipeline-stage-result-visualization-improvements/  and this earlier comment for details. if you have a request for a step that is related to the general title of the ticket but is not covered by the changes I mentioned, please file a new ticket along the lines of JENKINS-27092 .

            dnusbaum Devin Nusbaum
            leedega Kevin Phillips
            Votes:
            154 Vote for this issue
            Watchers:
            173 Start watching this issue

              Created:
              Updated:
              Resolved: