• Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Minor Minor
    • pipeline
    • None

      The xUnit plugin allows for a yellow "unstable" state, between red "FAIL" and blue "OK". This is useful for test-suite steps, to help developers immediately distinguish between "I broke one of 40,000 unit tests" and "I caused compilation to fail entirely".

      It would help if there were an easy way to assign an unstable state as the result of a step, especially a shell step (e.g. if we could check for a magic exit code to mark a step as unstable rather than failed)

          [JENKINS-26523] Allow yellow "unstable" state from a step

          Jesse Glick added a comment -

          The more commonly used JUnit plugin also has an UNSTABLE state—for the Run as a whole, which Workflow supports if the step wishes to set it. Jenkins generally just has a notion of a whole build being stable, unstable, or failed.

          Would rather leave this to JENKINS-26522 for marking up detailed information, and the existing option for a step to set a Run.result.

          Jesse Glick added a comment - The more commonly used JUnit plugin also has an UNSTABLE state—for the Run as a whole, which Workflow supports if the step wishes to set it. Jenkins generally just has a notion of a whole build being stable, unstable, or failed. Would rather leave this to JENKINS-26522 for marking up detailed information, and the existing option for a step to set a Run.result .

          Jesse Glick added a comment -

          I should add that FlowNode.getIconColor does exist, which renders the red or blue ball in the Running Steps view for example, but this is not based on a stored Result; it is red if there is an ErrorAction (exception of some kind), blue otherwise. So even allowing a step to set a custom result on its node would require at least a modest API change, such as a new ResultAction encoding a Result that getIconColor would use as an override, or a similar added option to ErrorAction.

          And to be useful, any new per-step status would need to be shown prominently. Running Steps would show it, but this is intended more as a detailed diagnostic view than something a user would normally be browsing. Visualizations like the stage view in Jenkins Enterprise do not show individual steps for the most part, so status would need to be set per stage to be useful.

          For the originally described use case, no new feature is necessary, since any integration of the xUnit plugin (presumably via SimpleBuildStep, like the JUnit plugin uses) would set the Run.result to UNSTABLE if there are test failures, which already captures the distinction between a compilation error halting the build early and one or more test failures recorded near the end. A per-FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors).

          Jesse Glick added a comment - I should add that FlowNode.getIconColor does exist, which renders the red or blue ball in the Running Steps view for example, but this is not based on a stored Result ; it is red if there is an ErrorAction (exception of some kind), blue otherwise. So even allowing a step to set a custom result on its node would require at least a modest API change, such as a new ResultAction encoding a Result that getIconColor would use as an override, or a similar added option to ErrorAction . And to be useful, any new per-step status would need to be shown prominently. Running Steps would show it, but this is intended more as a detailed diagnostic view than something a user would normally be browsing. Visualizations like the stage view in Jenkins Enterprise do not show individual steps for the most part, so status would need to be set per stage to be useful. For the originally described use case, no new feature is necessary, since any integration of the xUnit plugin (presumably via SimpleBuildStep , like the JUnit plugin uses) would set the Run.result to UNSTABLE if there are test failures, which already captures the distinction between a compilation error halting the build early and one or more test failures recorded near the end. A per- FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors).

          Jo Shields added a comment - - edited

          "A per-FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors)."

          I wish to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and the console log is inadequate to note this (even with specialized markup such as colors).

          Jo Shields added a comment - - edited "A per-FlowNode status would only be useful if you wished to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and if the console log were inadequate to note this (even with specialized markup such as colors)." I wish to have a number of different stages that could individually be marked as unstable or even failing without preventing subsequent stages from running and being displayed as stable, and the console log is inadequate to note this (even with specialized markup such as colors).

            jglick Jesse Glick
            directhex Jo Shields
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: