Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57801

Execution of post stage block is based on pipeline status, not stage result which docs says it should

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Blocker
    • Resolution: Unresolved
    • pipeline
    • None

    Description

      The declarative pipeline documentation on "post" says the following: "These condition blocks allow the execution of steps inside each condition depending on the completion status of the Pipeline or stage" 

      Currently, the post execution of a stage seems to refer to the whole pipeline status at that point and not the completion status of the stage as documented.

      Given the following pipeline:

      pipeline {
        agent any
        stages {
          stage('First') {
            steps { echo 'Hello' }
            post { success { echo 'First stage is success' } }
          }
          stage('Second') {
            steps { sh """ echo '<testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>' > test-results.xml """ }
            post {
              always { junit 'test-results.xml' }
              unstable { echo 'Second stage is unstable' }
            }
          }
          stage('Third') {
            steps { echo 'Hello' }
            post {
              success { echo 'Third stage is success' }
              unstable { echo 'Third stage is unstable! really???' }
            }
          }
        }
        post { unstable { echo 'Job is unstable' } }
      }
      
      

      I get the following output:

      [Pipeline] {
      [Pipeline] stage
      [Pipeline] { (First)
      [Pipeline] echo
      Hello
      Post stage
      [Pipeline] echo
      First stage is success
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (Second)
      [Pipeline] sh
      + echo <testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>
      Post stage
      [Pipeline] junit
      Recording test results
      [Pipeline] echo
      Second stage is unstable
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (Third)
      [Pipeline] echo
      Hello
      Post stage
      [Pipeline] echo
      Third stage is unstable! really???
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (Declarative: Post Actions)
      [Pipeline] echo
      Job is unstable
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      

      Clearly the third stage post unstable block is executed even though the stage itself is a success.

      This is also contradicted by the blue ocean stage view:

      Attachments

        Issue Links

          Activity

            allan_burdajewicz Allan BURDAJEWICZ added a comment - - edited

            That BuildCondition is what seem to be used whether you are in a stage or in the root pipeline.
            In that combined result the build result might always take precedence if it is worse.

            allan_burdajewicz Allan BURDAJEWICZ added a comment - - edited That BuildCondition is what seem to be used whether you are in a stage or in the root pipeline. In that combined result the build result might always take precedence if it is worse.
            per_bohlin Per Böhlin added a comment -

            I have not read the code, so I could very much be wrong in my assumptions, but based on the result presented below and more extensive tests that caused me to find the bug and find myself here in the first place, I wanted to present my observations - specifically for parallel stages.

            With this pipeline:

            pipeline {
                agent none
                options {
                        skipDefaultCheckout(true)
                }
                stages {
                    stage("Parallel stages") {
                        failFast false
                        parallel {
                            stage("1") {
                                agent {
                                    label 'linux'
                                }
                                steps {
                                    sh """ echo '<testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>' > test-results.xml """
                                }
                                post {
                                    always {
                                        echo "1: always"
                                        junit 'test-results.xml'
                                    }
                                    unstable {
                                        echo "1: unstable (from junit test result)"
                                    }
                                    success {
                                        echo "1: success"
                                    }
                                }
                            }
                            stage("2") {
                                agent {
                                    label 'linux'
                                }
                                steps {
                                    echo "Sleeping a bit to ensure stage 1 has finished even though running in parallel"
                                    sh 'sleep 3; exit 0'
                                }
                                post {
                                    always {
                                        echo "2: always"
                                    }
                                    unstable {
                                        echo "2: unstable"
                                    }
                                    success {
                                        echo "2: success"
                                    }
                                    failure {
                                        echo "2: failure"
                                    }
                                }
                            }              
                        }
                    }
                }
            }
            

            The output is (re-order interlacing for clarity):

            [1] + echo '<testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>'
            [1] 1: always
            [1] Recording test results
            [1] 1: unstable (from junit test result)
            [2] Sleeping a bit to ensure stage 1 has finished even though running in parallel
            [2] + sleep 3
            [2] + exit 0
            [2] 2: always
            [2] 2: unstable
            Finished: UNSTABLE
            

            If I cause stage 1 to fail with exit 1 after generating the test report I get:

            [1] + echo '<testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>'
            [1] + exit 1
            [1] 1: always
            [1] Recording test results
            Failed in branch 1
            [2] Sleeping a bit to ensure stage 1 has finished even though running in parallel
            [2] + sleep 3
            [2] + exit 0
            [2] 2: always
            [2] 2: success
            ERROR: script returned exit code 1
            Finished: FAILURE
            

            This makes me make the following "conclusions":

            The parallel stage collects the result from each individual stage and reports the aggregate (worst). Each stage is independent (being parallel) as they should be, with regard to the outcome of the steps. However, the post-actions does not modify the result of the stage it is associated with only, but also the "global" result. At least in the specific case of the junit test collection that change the "global" result to unstable in addition to making stage 1 unstable. After stage 2 steps runs successfully and enters post, one would assume that stage 2's status would be success. However, since unstable is triggered, that makes me believe that post actions not only modify the "global" result but also are triggered by the combination of the "global" result and the stage's steps' result.

            Then we have the additional case of the post-action not needing to change the stage's status. In my second example with failure in stage 1's steps, the junit post-action doesn't need to change the stage's result and by that also, does not affect the "global" result. Because, when we enter stage 2's post-actions, it runs as for success.

            In summary:

            Post-actions, in addition to modifying the status of its associated stage, it also affects a "global" state. The post-action trigger (always/success/unstable/...) is affected by the aggregate of its associated stage's steps' result as well as the "global" result from other post-actions. An alternative interpretation would be for the post-block to be the same instance for all stages, and hence gets a semi-global transient state across the stages.

            per_bohlin Per Böhlin added a comment - I have not read the code, so I could very much be wrong in my assumptions, but based on the result presented below and more extensive tests that caused me to find the bug and find myself here in the first place, I wanted to present my observations - specifically for parallel stages. With this pipeline: pipeline { agent none options { skipDefaultCheckout( true ) } stages { stage( "Parallel stages" ) { failFast false parallel { stage( "1" ) { agent { label 'linux' } steps { sh """ echo '<testsuite errors=" 1 " failures=" 0 " name=" test " skipped=" 0 " tests=" 1 " time=" 1 "><testcase classname=" " file=" file.txt " name=" name " time=" 0.000 "><error message=" error "></error></testcase></testsuite>' > test-results.xml " "" } post { always { echo "1: always" junit 'test-results.xml' } unstable { echo "1: unstable (from junit test result)" } success { echo "1: success" } } } stage( "2" ) { agent { label 'linux' } steps { echo "Sleeping a bit to ensure stage 1 has finished even though running in parallel" sh 'sleep 3; exit 0' } post { always { echo "2: always" } unstable { echo "2: unstable" } success { echo "2: success" } failure { echo "2: failure" } } } } } } } The output is (re-order interlacing for clarity): [1] + echo '<testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>' [1] 1: always [1] Recording test results [1] 1: unstable (from junit test result) [2] Sleeping a bit to ensure stage 1 has finished even though running in parallel [2] + sleep 3 [2] + exit 0 [2] 2: always [2] 2: unstable Finished: UNSTABLE If I cause stage 1 to fail with exit 1 after generating the test report I get: [1] + echo '<testsuite errors="1" failures="0" name="test" skipped="0" tests="1" time="1"><testcase classname="" file="file.txt" name="name" time="0.000"><error message="error"></error></testcase></testsuite>' [1] + exit 1 [1] 1: always [1] Recording test results Failed in branch 1 [2] Sleeping a bit to ensure stage 1 has finished even though running in parallel [2] + sleep 3 [2] + exit 0 [2] 2: always [2] 2: success ERROR: script returned exit code 1 Finished: FAILURE This makes me make the following "conclusions": The parallel stage collects the result from each individual stage and reports the aggregate (worst). Each stage is independent (being parallel) as they should be, with regard to the outcome of the steps. However, the post-actions does not modify the result of the stage it is associated with only, but also the "global" result. At least in the specific case of the junit test collection that change the "global" result to unstable in addition to making stage 1 unstable. After stage 2 steps runs successfully and enters post, one would assume that stage 2's status would be success. However, since unstable is triggered, that makes me believe that post actions not only modify the "global" result but also are triggered by the combination of the "global" result and the stage's steps' result. Then we have the additional case of the post-action not needing to change the stage's status. In my second example with failure in stage 1's steps, the junit post-action doesn't need to change the stage's result and by that also, does not affect the "global" result. Because, when we enter stage 2's post-actions, it runs as for success. In summary: Post-actions, in addition to modifying the status of its associated stage, it also affects a "global" state. The post-action trigger (always/success/unstable/...) is affected by the aggregate of its associated stage's steps' result as well as the "global" result from other post-actions. An alternative interpretation would be for the post-block to be the same instance for all stages, and hence gets a semi-global transient state across the stages.
            teilo James Nord added a comment -

            The documentation is incorrect. 

            Currently success and failure can look at the individual stage status.  Aborted, changed, unstable can only look at the pipeline status.

            teilo James Nord added a comment - The documentation is incorrect.  Currently success and failure can look at the individual stage status.  Aborted, changed, unstable can only look at the pipeline status.

            teilo I'm not quite sure how to interpret your previous comment in the context of my not wanting one failed (or unstable) stage in a parallel group to make other successful stages in the same parallel take their not their post-.{failed,unsuccessful} actions rather than taking their post->success actions.

            IOW, every stage in a parallel group should take the post action path for their own stage.  Not the worse of the parallel group, or the worst of the job.  I just cannot fathom how either of those latter cases make any sense at all.  The post action stages for an individual stage are going to deal with the status of that stage and having them deal with it incorrectly because of some higher level aggregate status, again, makes absolutely no sense.

            If one wants post actions to happen on the aggregate of a parallel group, or the whole pipeline, one can add post actions at those levels.

            brianjmurrell Brian J Murrell added a comment - teilo I'm not quite sure how to interpret your previous comment in the context of my not wanting one failed (or unstable) stage in a parallel group to make other successful stages in the same parallel take their not their post-.{failed,unsuccessful} actions rather than taking their post->success actions. IOW, every stage in a parallel group should take the post action path for their own stage.  Not the worse of the parallel group, or the worst of the job.  I just cannot fathom how either of those latter cases make any sense at all.  The post action stages for an individual stage are going to deal with the status of that stage and having them deal with it incorrectly because of some higher level aggregate status, again, makes absolutely no sense. If one wants post actions to happen on the aggregate of a parallel group, or the whole pipeline, one can add post actions at those levels.
            trunov_ms Max Trunov added a comment - - edited

            >>These condition blocks allow the execution of steps inside each condition depending on the completion status of the Pipeline or stage.

            But NOT

            This issue main cause that we still use the scripted pipeline. We don't have "stage" state.

            We have GLOBAL "build" state only and every "local" post(for stage) use it.

             

            trunov_ms Max Trunov added a comment - - edited >>These condition blocks allow the execution of steps inside each condition depending on the completion status of the Pipeline or stage. But NOT This issue main cause that we still use the scripted pipeline. We don't have "stage" state. We have GLOBAL "build" state only and every "local" post(for stage) use it.  

            People

              Unassigned Unassigned
              dcendents Daniel Beland
              Votes:
              7 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: