A sample response json is:

{
"_class": "io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeImpl",
"_links": {},
"actions": [],
"displayName": "Checkout",
"durationInMillis": 8550,
"id": "5",
"result": "UNSTABLE",
"startTime": "2016-10-24T10:37:04.263+1100",
"state": "FINISHED",
"edges": [
{
"_class": "io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeImpl$EdgeImpl",
"id": "7"
}
]
}

This is the first stage, which should have succeeded.

svanoort ideally we'd only like to show the specific stage that is unstable when the pipeline is unstable (not all stages). Is that feasible?

Its going to be fixed in JENKINS-39296, closing as duplicate.

This is not a duplicate, or else that other one does not fix this.

michaelneale Sorry, yes you are right. What I fixed is all steps showing it's own status.

yes this one isn't blue ocean specific either it seems.

svanoort Bismuth API returns all stage's status as run's status - that is if the build is completed.

StatusAndtiming.computeChunkStatus(@Nonnull WorkflowRun run,
                                                   @CheckForNull FlowNode before, @Nonnull FlowNode firstNode,
                                                   @Nonnull FlowNode lastNode, @CheckForNull FlowNode after)

I am using this method to compute status of chunk(stage). This method however does not seem to be doing the right thing. What I expect to do is to compute status of a chunk by inferring status based on the steps run inside it.

See https://github.com/svanoort/pipeline-graph-analysis-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/pipelinegraphanalysis/StatusAndTiming.java#L158.

My blueocean specific (pre-bismuth) implementation used logic to compute status of stage or parallel using underlying steps and last node of parallel branch and blocked stage.

• stage/parallel is FAILURE if at least one of more node inside it failed or last node of chunk had error.
• stage/parallel is RUNNING if at least one step inside it running
• stage/parallel is FINISHED if no steps inside it are running
• Else if FINISHED and no error on any step or last node on the chunk had no error it was SUCCESS

There doesn't seem to be a way to get ABORTED status on step so we can't really compute ABORTED stage. For all error pipeline API gives AbortException so there is no way to distinguish user aborted vs script FAILURE.

For UNSTABLE result, is there way to get step level UNSTABLE state?

Anyways, looks like I can't really use StatusAndTiming status APIs to get status correctly. Is this something you plan to implement, otherwise I will have to take care in blueocean code?

@vivek This is the expected and documented behavior - https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Stage+View+Plugin#PipelineStageViewPlugin-1.6%28July18%2C2016%29

There's no way for the pipeline to identify which step set the build status to unstable, so we have to assume it was one of the otherwise successful steps.

> There doesn't seem to be a way to get ABORTED status on step so we can't really compute ABORTED stage. For all error pipeline API gives AbortException so there is no way to distinguish user aborted vs script FAILURE.

If your step is the final step of the build and the build was aborted, then your step is aborted. Otherwise you are correct: there's no way to identify a failed versus manually aborted step. I'm not sure why you're saying that the status and timing APIs are the problem here – how would you propose I work around the data model limitation?

> stage/parallel is FAILURE if at least one of more node inside it failed or last node of chunk had error.

No, a stage/parallel is a FAILURE if the final node in it had an error – otherwise the error was handled in a try/catch block. This was the same mistake that legacy Stage View made. Unfortunately jglick shot down my proposal to properly track this status in pipeline since this is not the best way to handle this.

I've told jglick time and time again that our handling of status coding for steps is grossly inadequate, but still he closes proposals to add an unstable status to a step.

The right way to handle this is definitely JENKINS-26522 + step-level support for its custom statuses, but unfortunately that has been blocked by Blue Ocean itself.

There's no way for the pipeline to identify which step set the build status to unstable

Well, if the script just ran currentBuild.result = 'UNSTABLE' or a step called Run.setResult that is true; we would need a proper implementation of JENKINS-26522. (I am only opposed to partial implementations which do not actually update at least one UI to use the capability.) More likely we would want JENKINS-27395, which is what would accomplish the actual goal specified in the initial issue description here. In fact I think this is just a duplicate. Again the issue is not implementing the step & data model, which is trivial, but deciding which UIs should interpret that data and how.

For the case of ABORTED, typically this is set not due to a call to Run.setResult inside the script, but rather the program as a whole terminating with FlowInterruptedException. (That API supports any Result, but ABORTED is the most common.) In such cases you can tell where the exception was thrown by inspecting ErrorAction.

kzantow I would point out that this idiom from the master branch makes no sense at all. You could just delete the try and finally blocks and the result would still be FAILURE—but more easily analyzed.

Thanks svanoort.

>I'm not sure why you're saying that the status and timing APIs are the problem here – how would you propose I work around the data model limitation?

I didn't say StatusAndtiming is a problem. What I meant was, these API doesn't give how blueocean wants to repot per state status. For example, if overall run is UNSTABLE, this API will give UNSTABLE for all steps. Where in blueocean we want to repot per set status by computing if that step had any error or it's still running or not.

>No, a stage/parallel is a FAILURE if the final node in it had an error – otherwise the error was handled in a try/catch block. This was the same mistake that legacy Stage View made. Unfortunately Jesse Glick shot down my proposal to properly track this status in pipeline since this is not the best way to handle this.

Yes, makes sense.

>The right way to handle this is definitely JENKINS-26522 + step-level support for its custom statuses, but unfortunately that has been blocked by Blue Ocean itself.

I guess I need more context here, just so that blueocean can unblock it

jglick

For the case of ABORTED, typically this is set not due to a call to Run.setResult inside the script, but rather the program as a whole terminating with FlowInterruptedException. (That API supports any Result, but ABORTED is the most common.) In such cases you can tell where the exception was thrown by inspecting ErrorAction.

Thanks, that is a proposal that might solve that aspect. I think my concern is that we're building up a set of fairly complex rules now for what is the "status" of a FlowNode. Rather than doing that, we should just add a status coding (probably JENKINS-26522) and have steps set it where needed – we can cut out so much complexity very fast, and give more data and control to the user.

> Where in blueocean we want to repot per set status by computing if that step had any error or it's still running or not.

So, we still do this with the StatusAndTiming APIs – if you want to hide the uncertain UNSTABLE assignment until there's a better mechanism for attributing it, one solution is to coerce UNSTABLE status to SUCCESS (completed) in your mapping (always legal IIRC).

> I guess I need more context here, just so that blueocean can unblock it

Primarily that Stage View would be the logical case to demo a new JENKINS-26522 API in the UI and we don't do enhancements to it generally now. However if we just did explicit status coding with this and not tags / other metadata, then that's purely a change to pipeline-graph-analysis plugin.

svanoort no https://issues.jenkins-ci.org/browse/JENKINS-26522 would not fix it - it requires the users to do some work (likely after they have a build with test failures) - unless you mean that test reports could also use the same facility? If so (or if currentBuild.result setter would do it) then yes it might. There would be no GUI change on the blue ocean side needed to accomodate this.

jglick no I don't think this is a duplicate of having multiple test results on multiple stages (https://issues.jenkins-ci.org/browse/JENKINS-27395) - this is just about unstable status (there may or may not be tests involved, of course most of the time there is so...).

If JENKINS-26522 was implemented, there would be no GUI change required on blue ocean at least for it to work.

there may or may not be tests involved

There is not going to be any change to the semantics of setting currentBuild.result. That sets the overall build result, period.

The (vague) proposals being floated would allow the junit step to (somehow) include a test set identifier which would “scope” the results; and/or to allow a user to explicitly mark status (and other things—badges, comments, …) on some block, probably a stage. IIUC neither would affect the behavior of existing scripts: you would need to opt in somehow.

> There is not going to be any change to the semantics of setting currentBuild.result.

This has no relationship, because it completely bypasses the flow graph to directly modify the run result (making it something of a dirty hack). JUnit would only overlap with this slightly – because it would set an explicit step status.

Users want a way to set more discrete and more complex statuses on steps. Blue Ocean wants a way to do this. Rather than trying to hold back the tide, let's talk concrete proposals for how to make this happen: I'm not sure where jglick is getting "vague" from but I've laid out a concrete implementation proposal and a couple of proposed syntaxes in JENKINS-26522.

michaelneale I propose we close this one as duplicate and shuffle discussion there. Objections?

michaelneale To clarify: JUnit would set the explicit status coding via JENKINS-26522 on the appropriate step(s) - UNSTABLE for example. Then the status-determining logic in pipeline graph analysis plugin would pick this status coding up and apply it to appropriate step(s).

Thanks svanoort jglick that explanation makes sense to me, I think I get it now.
So yes, in that case, closing this in favour of JENKINS-26522

We are going to reopen this so the Blue Ocean team can track JENKINS-26522 and verify that it works as expected when it is completed.

Sorry for the updatespam – I had this confused with another similarly named bug/RFE that this appears to clone. In any case, jamesdumay I would not expect this to be a trivial change since you need to rework the way statuses are tracked.

Basically, you can increase priority as much as you like, but it needs a significant data model change to do it the right way. If you choose to do it the wrong way, that will have to be purely Blue Ocean side and supported, and we can make no guarantees it will remain functional in the face of future pipeline changes. If you do it the right way, it will take a while though.

Thanks svanoort! I understand this one is a difficult one to implement and its going to take its time. Im just adjusting tickets that are important to Blue Oceans 12 month roadmap.

Is it really only depending on Blue Ocean? We have the exact same behaviour in the legacy view.

All stages are shown unstable while only one step failed a specific stage (rending unstable). In the other hand, the stage that failed the build is shown "failed". Is it a different issue? 

shaiton no it doesn't really - its part of the model of pipeline, so not something blue ocean can solve (as you have guessed)

Any updates on this issue? From a develop's perpective it's weird to see all build steps unstable when there is only one step failing. So it's more time consuming as they need more time to identify where the problem occurred.

mdelapenya nothing to try yet - but most of the work is reworking pipeline to have fine grained statuses (which is non trivial, touches a lot of code). The UI side is relatively trivial once that is done. 

Testing Notes

• From the UI side, once implemented, automated test(s) using a predefined set of Pipelines would be valuable. The test could include the use of "unstable," "successful," and "only-stage-seven-fails" pipelines, for example, and verify that the correct statuses are reported.
• From the Pipeline side, unit tests should be made part of the PR. Considering the folks working on this change, I've got every confidence that it'll be well tested in this way.

This is a serious issue we have with the blue ocean and the legacy view, to the point that developers are asking to go back and not use "pipeline".

It would be really helpful if you can solve this issue.

eparimi You can always manually set the build result to SUCCESS rather than UNSTABLE (see: https://support.cloudbees.com/hc/en-us/articles/218554077-How-to-set-current-build-result-in-Pipeline-). 

abayer Has been working on this on and off for many months, because it actually requires adding a complex set of new features rather than just a simple bugfix, and we don't have a consensus on how to best support this.  We basically have to radically change how statuses are determined.

svanoort As far as I know, It's not possible to recover from an UNSTABLE state using currentBuild.result:

node {
  stage('Stage 1') {
    echo 'foo'
    currentBuild.result = 'SUCCESS'
  }
  stage('Stage 2') {
    echo 'foo'
    currentBuild.result = 'UNSTABLE'
  }
  stage('Stage 3') {
    echo 'foo'
    currentBuild.result = 'SUCCESS'
  } 
}

Results in UNSTABLE status for the whole pipeline, and all the stages appearing yellow colored.

luispiedra I don't think we are going to get a workaround for this. The work here to fix statuses needs to happen so it all works properly.

Hello,

Any updates on this issue plz? It's weird to see all build steps unstable when there is only one step failing.

This is a serious issue we have with the blue ocean view

hbedoui we still plan on fixing this but no ETA yet. Requires a big rework of how statuses are reporting in Pipeline, so its not as simple as it might seem.

So, we've been slowly adopting the new Pipeline APIs for all of our production builds across dozens of development teams. This build status issue is causing a very large amount of frustration and wastes a lot of developer time trying to dig through logs to deduce failure statuses. As such I'm hoping this is being considered a high-priroity item to be fixed regardless of how difficult it may be to do.

That small rant aside, might I suggest that if the "correct" fix for this turns out to be something incredibly complex and difficult - which, again, suggests that it may be months or years before it is fixed - could you not find some reasonable substitution or workaround for the problem in the meantime? Like, for example, could you perhaps create a clone of the `stage()` function, say, `manualStatusStage()`, that maybe ignores all of the current build status logic completely but allows the build status to be explicitly set using a custom variable or method, something like `currentBuild.hackStagedResult = 'SUCCESS"`? In this way users that are experiencing this pain point would have some temporary workaround to ease the current pain while allowing you some time to fix the underlying problem correctly.

NOTE: I'd even be happy if these "hacks" or "workarounds" were simply undocumented features that you put in place as temporary workarounds to this specific JIRA issue, with the expectation that once the underlying problems are fixed the "undocumented" functionality will simply be removed - and hence, no need to be maintained going forward.

Thoughts?

leedega Believe me, we know exactly how irritating these limitations of the Pipeline architecture are because we "drink our own champagne" internally and use test result reporting extensively in Jenkins Pipelines. The problem is that this is directly tied to the limitations of the Pipeline data model, which internally is pass/fail – everything else is computed in a logical way on the basis of ErrorActions, Input Actions, whether or not the FlowNode in question is the tip of a incomplete build (In-progress).

Anything that modifies how we store status is unfortunately a permanent change because it would have to work by recording data in the model, either by Actions on FlowNodes, or by some internal state, or both. Thus any "hacks" we added would be a part of Pipeline forever – we could ignore the extra data but it would introduce compatibility issues.

The extra level of complexity is that any behavior needs to be consistent across (potentially nested) parallels, nested blocks, and has to have something like a try/catch construct to let users override results – which has to behave logically in the face of a full & potentially complex graph structure. Take a look at Pipeline Graph Analysis plugin and you'll see what I mean about the complexity (also the APIs for working with the graph in https://github.com/jenkinsci/workflow-api-plugin/tree/master/src/main/java/org/jenkinsci/plugins/workflow/graphanalysis).

The simple fix would be to simply not display an UNSTABLE status at the UI level in any situation for Pipelines, only at the overall build level. Would that be preferred? Because that's a change we can do without too much extra hassle.

Then your architecture is wrong. It cannot support something fundamentally needed. The sooner you change it, the better.
This project goes nowhere without this.

leedega Perhaps someone (you or someone else) can implement such a “hack” as a plugin – probably with a fat disclaimer indicating that many of the scenarios (as mentioned by svanoort) are not covered.

That would at least reduce the pain a bit, and it would “buy some time” to get the real solution in place.

svanoort: Your suggestion to 

[…] simply not display an UNSTABLE status at the UI level in any situation […]

lqbweb It is a high priority to address this and in my personal upcoming work, but right now the highest priority is ensuring Pipeline does not cause system stability problems.

As far as the architecture: I took the time to write up a detailed explanation of the situation in hopes it might lead to something productive (maybe even a PR), and it really is not constructive to simply say "your architecture is wrong." While you're entitled to your opinion, let me ask you this: how would you feel if you were putting many of your nights and weekends into improving something and someone said that to you? This is an open source project and we only have so many engineers no matter how hard we work – if you think you could improve upon the architecture you're more than welcome to submit PRs that accomplish this goal.

znerd

Perhaps someone (you or someone else) can implement such a “hack” as a plugin – probably with a fat disclaimer indicating that many of the scenarios (as mentioned by Sam Van Oort) are not covered

I was actually going to suggest that. Some of those experiments can provide new ideas for how to implement a comprehensive feature – and if they work really well they may even be requested to directly merge into the core of Pipeline. More often they provide inspiration that leads one of us to hack out the ultimate solution.

Personally, I absolutely hate that we can't easily address this improvement, but past lessons have also taught us that a maintainable and robust solution is much better for this kind of challenge than a quick-and-dirty solution that opens up problems down the road. Maybe someone will be able to come up with a solution that does not impose those risks though.

 This build status issue is causing a very large amount of frustration and wastes a lot of developer time trying to dig through logs to deduce failure statuses

leedega we have had the same issue. Today we work extensively with Slack notifications that contain links to html reports (this is more important than just showing the correctly coloured box anyway, because even with that the developer would still have to check the log file for the actual root cause). 

In general I think it is great to have a per-pipeline-run build status - this probably shouldn't even be refactored: There are quite some use cases that need that: Notifications to source control if a feature branch can be merged, notifications about "back to normal", just showing a color in the build list in the left panel. That said I think it would be great to have additionally a per stage status, but not only for success/unstable/failed but with a much richer data model - it could e.g. be a list of info, warn and error messages that can be reduced to info/warn/error counts and even more reduced to success/unstable/failed per stage. If that was implemented as stage.info(...), stage.warn(...), stage.error(...) it could under the hood automatically set the per-pipeline-run build status. Plugins could without hassle one by one switch to the new way of setting the build status (while the old way stays intact).

All that being said, if for a certain use case it is really important to only "change the color of one box", there is currently one way to achieve it:

stage('Stage 1') { echo "green" }
try {
  stage('Stage 2') { throw new IllegalStateException("a failed stage") }
} catch(err) { echo "made the stage only red" }
stage('Stage 3') { echo "green" }
try {
  stage('Stage 4') {  throw new IllegalStateException("a failed stage") }
} catch(err) { echo "made the stage only red" }
stage('Stage 5') { echo "green" }

Will get you

svanoort As people's grief lies mostly in the UI presentation as it seems (and not in the underlying data model), maybe it could be a very simple change in the pipeline core plugin to show yellow for a predefined exception?

ghenzler Is it possible to do the same in declarative pipeline?

I'm failing to understand how this issue is any less than Critical.  It completely invalidates the result of any and every pipeline job run that has even the smallest subset of stage failures.   The problem even propagates outside of Jenkins and Blue Ocean as it leads to other plugins such as the Pipeline Githubnotify Step Plugin incorrectly setting FAILURE status on GitHub PR commit statuses set from stages in the pipeline.

This issue leads to developers having to waste time and drive up frustration having to hunt through the stages of their pipeline job runs to figure out which stage actually did fail.

Additionally, it's really disheartening to see that this issue is only as far as Planned on the Blue Ocean Roadmap. Surely the 116 (or more) votes and 143 (or more) watchers on this issue demonstrates how important it is for a lot of people.  Other items on the roadmap that are In Progress even, such as JENKINS-44699 don't have anywhere near as many votes and/or watchers.

Is there any ETA, no matter how much of guesstimate it is, available as to how long this issue will take to be fixed?

This critical issue is maddening. It's almost as if we have to separate our pipeline into small components (jobs) like we did with freestyle jobs, for the build status reporting to make sense.

Indeed yes, it is critical. This has somehow slipped. It was looked at some time ago, but unfortunately is a HUGE amount of work as there is no way in the pipeline engine to track this information (especially not if people are using scripted pipelines, which are still popular, and were included at the time). It is not at all a small job

cc vivek jbriden this somehow has slipped off the radar. It was on the old roadmap but don't know the status of it now. Given how old it is, it may be more feasible to do this in a declarative only context now and actually be possible. 

ghenzler - Thanks for the suggestion on using IllegalStateException to turn single stage red. Does this work for the Blue Ocean UI as well?

michaelneale - Are you suggesting that the scope of this issue be narrowed to declarative syntax? I would imagine that many of the voters and watchers of this issue are using scripted pipelines. I know we are, as there are many advanced scenarios that simply are not feasible using declarative syntax.

While I can appreciate the desire for a single solution to address both Pipeline styles, if a solution for one of the styles is relatively easy while the other quite complex, let's not let the perfect be the enemy of the good here.  Please.

Fully agree with brianjmurrell. But then should we split this ticket in two? Then…

• …one of them (declarative) could be picked up and resolved,
• …while the other one (scripted) could potentially be implemented on a different schedule.

An additional benefit could be that when we've got the solution in place for the declarative pipeline, the challenge for implementing the fix for scripted could be reduced, because we already have a guiding implementation in place (the one for declarative).

I do not see any reason to think that this issue is specific to Declarative vs. Scripted. Nor do I think it is necessarily a “huge amount of work”. There is already an API to indicate that a step failed: ErrorAction. This is added implicitly when StepContext.onFailure is called. The overall build status is set to failed (or some other statuses, acc. to FlowInterruptedException) if the FlowEndNode is marked this way, typically because the error was not caught and handled.

To set a less severe result, there is a StepContext.setResult API. Unfortunately the implementation in CpsStepContext simply sets an overall build status, which is not useful, and anyway JUnitResultsStepExecution does not call it. There could perhaps be a new action type retroactively implemented by ErrorAction, or somehow related to it, which would be added to a FlowNode when setResult is called and which just defines a getResult method.

This would allow JUnitResultsStepExecution to set a per-step UNSTABLE status instead of a build result, at least optionally. (I think this would need to be opt-in for compatibility reasons, unless you declare that setting a step result also makes the build result at least as bad, which may be unwanted.) Then FlowNode.getIconColor would honor the new action, causing Pipeline Steps in the classic UI to show yellow balls where appropriate, and Blue Ocean would need to be patched to do the same.

Is any progress being made on this issue?

I don't want to sound impatient but this is a pretty seriously debilitating issue!  Way beyond even just the graphical display of an entire pipeline of stages and steps looking like it's unstable, this issue completely defeats any and all {{post { }}} block processing after the first stage failure.  Every stage block from the point of a failed stage on gets processed as unstable, preventing one from doing any kind of finer (than entire pipeline) granulated processing and/or reporting of pipelines.

I now personally believe this issue to be higher than Critical and believe it to be an outright Blocker as it makes much of the usefulness of Pipelines void.  It makes me want to break my pipeline down into separate free-style jobs just so that I can get meaningful reporting and processing of jobs based on whether they are successful, unstable, failed, etc.  A bug serious enough to make one want to abandon Pipelines is by definition a Blocker to Pipelines.

So, can we get a status update on this?  Is anyone actively working on it or has it been thrown back on the pile?

Same here. Situation with post actions is very annoying for us. It's broking all logic of pipeline.

brianjmurrell zaharovdi Short term, what we can do is roll back the support for UNSTABLE status. That means you never have to worry about this "bug" because we don't try to signal that a stage somewhere set that status – we simply go back to ignoring it entirely. What do you think?

In the longer-term, we're building a new Pipeline engine that will replace CPS and provide solutions to a lot of painful issues (sandboxing, lack of an integrated debugger, stability and resource-use problems) and we're planning to address the way we store Step statuses there so that this ceases to be a problem. However I can't give you a firm timeline for that yet.

For those who will ask "Why isn't this super easy? Why can't it be done tomorrow?!" – the answer is: "There's nothing actually storing when we set unstable status. And storing that is deceptively complex because statuses can interact and carries a huge risk of regressions." In fact, all the statuses besides success and failure are computed based on a pretty complex and delicate process. Much, much, much more is happening under the surface than users will actually see.

abayer Would it be viable to add a custom "UNSTABLE OR HIGHER" post condition for the Declarative post conditions? I think that would resolve the part of this that seems to be serious (breaking post behaviors that expect success only).

zakharovdi brianjmurrell I've opened JENKINS-53889 which I think would resolve your biggest issue here by making it easy to catch and handle UNSTABLE statuses in declarative.

Short term, what we can do is roll back the support for UNSTABLE status. That means you never have to worry about this "bug" because we don't try to signal that a stage somewhere set that status – we simply go back to ignoring it entirely. What do you think?

svanoort What is the value of currentBuild.currentResult at various stages with this proposal? Say I have 3 parallel stages A, B, and C and A is SUCCESSful and B is UNSTABLE and C is SUCCESSful. Will I be able to determine in stage C that it is SUCCESSful even though stage B was UNSTABLE? Because that is what currently doesn't happen. Currently, as soon as B hits UNSTABLE, every stage after it is UNSTABLE even if it's not really.

brianjmurrell currentBuild.currentResult would be entirely unchanged. It has no representation within the Pipeline data model (it directly modifies the build). What this would do is ensure that no stages show up as "unstable" for you. Which gives you what you are asking for.

If you want the ability to keep the UNSTABLE status and figure out which stage actually set it... well, see above, that's way more work than one would reasonably expect (and much riskier too). Plus it requires reworking steps that can generate that status (generally steps that process unit test results, so JUnit and others).

svanoort So, no then, your proposal doesn't really help at all, unless there is an angle that I am missing.

So to repeat the use-case:

Say I have 3 parallel stages A, B, and C and A is SUCCESSful and B is UNSTABLE and C is SUCCESSful. Will I be able to determine in stage C that it is SUCCESSful even though stage B was UNSTABLE?

And to further clarify further, the stages are parallel but A completes before B and B completes before C.

I have to be able to determine in C that C was in fact successful. And I really only need to be able to do this because of this whole issue. Because the pass/fail stages cannot be determined any other way by the user, I have to post comments to the PR about which ones actually were UNSTABLE but I cannot even reliably do that if B's state is going to pollute C's state!

brianjmurrell Unfortunately, that and the proposed mitigation in https://issues.jenkins-ci.org/browse/JENKINS-53889 are all we've got as viable short-term solutions. It's not a matter of "polluting state" - the data Pipeline needs to figure out which of the stages set the UNSTABLE status simply isn't there, and injecting that into the current system will probably break it completely.

Several other engineers and I are working on a comprehensive solution to the things that make this problem hard to solve. We've already had several top Jenkins engineers spend literally months going in circles working on find a solid implementation that solves this specific issue without causing major regressions – the pull requests are out there if you're curious. Unless someone comes up with an unexpectedly clever solution, what's left is waiting for that larger effort to wrap up and then this specific problem will be solved along the way.

And that sucks, and we know it's frustrating waiting, but we're doing the best we can to get that to the point of an alpha release.

Following jglick's advice in his comment, I have digged a bit deeper into this issue and found a way to display what stages actually failed that works today without code change (still all stages show up as UNSTABLE, but the ones that have failed are marked as failed):

stage('Build') { }     
for(int i=1; i<=7; i++) {
    try{ 
        stage('Test Suite '+i) { 
            if(Math.floor(Math.random() * 3) % 3 == 0) {
                throw new RuntimeException("tests failed, you would use a custom exception type here")
            }
        }
    } catch(/* your custom exception type here */ err) { currentBuild.result = 'UNSTABLE' }    
}

This creates a stage view as follows:

This works because the pipeline-stage-view-plugin [1] takes the exception markers into account per stage. I'm sure blue ocean could also easily show this information.

[1] https://github.com/jenkinsci/pipeline-stage-view-plugin/blob/47101e971da4d16a08a4a427c3d72f92cec8ff53/ui/src/main/js/view/templates/pipeline-staged.hbs#L100

ghenzler, I currently use FlowInterruptedException to achieve the same thing. The added benefit is this exception has a Result field.

I've submitted pipeline-graph-analysis-plugin#PR22 to use the FlowInterruptedException's Result to determine the Stage status rather than using FAILURE (pipeline-graph-analysis-plugin 1.7) or ABORTED (pipeline-graph-analysis-plugin 1.9).

Dear all,

I want to add here a user experience.
The team for which I setup the CI are really in a trouble when they see a totally yellow pipeline. They don't want to make the effort to check each stage one by one to find which is OK and which is not.
I can understand that, and .. .well... users or customers are always right, isn't it ?
I am afraid that the proposed stage view, with a simple red note will not help. Even if a green note is added for stage with status SUCCESS.

In such a situation, have in mind that users don't want to adopt the pipeline. It is a real show-stopper. And they are currently asking me to come back to something which breaks the pipeline concept.

So, I don't know what is the level of complexity, but to have green (or blue) color for stages ended with success, red for NOK ones and yellow for Unstable stages would be nothing else than a must have. That will really be user friendly and avoid headache for CI guys that must battle against upset users .

This ticket now has 149 votes and 169 watchers, up from 116 and 143 only 4 months ago.  Needless to say that it's affecting a lot of people.

It's also very disheartening to see such a critical issue has not made it beyond the same Planned stage on the Blue Ocean Roadmap that it was back at the beginning of September.  It's not even In Progress yet, 4+ months later.

Does the Jenkins team really think this issue is that unimportant?

I understand that it's complicated, but is that reason enough to not even get started on it?

Was so excited to move our production builds to using the pipeline and blue ocean visualization. With all the stages changing from green to unstable, it's a blocker. Even posting each stage status to slack becomes misleading, in that healthy stages post unstable, and there are instances of unstable stages posting as success, that's a deal breaker right off the bat.

Can I suggest a short term workaround?

If there was an option in the pipeline: "stageStatus=independent" (or something like that), which worked this way, it would allow us to have control and use blue ocean until it's resolved:

• simply allow each stage icon to use the color/state of it's own independent activity

That's it. I'm not sure if that's a flowNode or what, but whatever each icon is in any stage, they all seem to be capable (today) of executing and knowing if that execution was successful or unstable or failure or aborted. If that can simply be retained and displayed for that icon, this works for me. As far as what the overall build status ends up as in this case – if you allow us to set it at the end of the pipeline, we could code that to our own needs, based on the individual statuses per stage.

This ticket has added 26 more votes and 20 more watchers in just a hair over a month for a total of 175 and 189, respectively.

Could somebody please provide an update on the status of this issue to those 189 watchers please?

Is this really (after all of this time and attention) still just in the Planned state that the  Blue Ocean Roadmap indicates?

this is necessary to enhance the pipeline status view, to be able to mark each stage status separately, to identify which of them if passed, unstable or failed. 
It is should not be Jenkins's decision to abort the pipeline or not. 
what if there are independent stages?
what if I want to continue and just be aware about test/stage that was failed?
I think the status and the abort action should be performed explicitly. 
for example - stage function that received 3 arguments:
1 - stage status (mandatory)
2 - Boolean if need to abort (optional - default to continue)
3 - message to be print in the stage view window (optional)

it is will be very helpful.   

Does the Jenkins team really think this issue is not important?

I understand that the logic associated with the overall job status is established throughout Jenkins and that's difficult to adjust. (at least that's what I've heard the problem is, re: 'fixing' this). But in the meantime, can't we just leave that logic alone, and create a new property for the stage object, called 'stage status', that is limited in scope to the execution of that specific stage, and give the user an option to use legacy behavior, or display per-stage-status? The 'overall' job status logic could remain the same.

To ignore the entire user base screaming about this about to drive me away, and is so different from the great experience and responsiveness I've had with the Jenkins team over the years.

added an attachment 'allYellow.jpg' to illustrate the impact. The various grouping of 12 stages on the right of the pipeline are all test related. Before the pipeline got that far – everything was green, as all those previous stages were successful. And for the 12 test related stages on the right... 2 of them had some failures, the other 10 had no failures. But examining a visualization to get a good understand of what went well and what had issues.. all value is lost, nothing has meaning.

Completely agree with borisivan.  To the extent even that I have written Blue Ocean off.  It causes more questions and confusion that it adds value.

Instead, I tell my users that GitHub's commit statuses are the point of truth and to that end provide a status for every pipeline stage complete with a URL to (non-Blue Ocean) stage results in the Details link of each GitHub status posted.

Hi Team,

Did anyone found any solution for Declarative Pipeline?

Please share details if any solution will be there.

Thanks,

Vaishal

this one requires a rewrite of a chunk of pipeline engine - abayer is it fair to say that this is part of new generation ?

I don't see this changing for declarative (given all things use "currentBuild." which can't work with this by design) - so I am not sure if this will be done in this form. 

From an outside perspective, this is ridiculous. It cannot be that hard to determine and save the result of a single node and color it accordingly.

Yes it would seem odd from outside - but from inside it kind of makes sense. Plugins mark the whole run as unstable, not a stage. This is a pipeline thing misclassified as blue ocean (other visualisations do the same). 

If a build failed then stopped - could just show the "last" stage as unstable only - but if it continues on (which some do by default) then that may be misleading as it may have been an earlier stage that showed it up. 

I have a somewhat related question:

Is there an API to access the results of the various stages in a pipeline? I have been playing tricks like setting a variable with the name of the current stage when a stage starts and checking its contents in the post for the pipeline to report which stage failed and would like to have a cleaner solution to work with what is already in place...

nancyrobertson2005 I think you hit the nail on the head - I am not sure such a facillity exists. If it did - things like this would be possible. 

I can understand the existing logic and methodology makes this difficult to change.
But I don't understand why something simple like this couldn't be done as a workaround:

• add a new property (doesn't exist today) to each stage object: stageIndependentStatus
• when each stage completes, immediately save the value of the independent execution of that stage in stageIndependentStatus

This way, you can let all the existing logic stay as-is regarding stage state, state propagation to other stages, etc – all the things that are difficult to change. The "overall build" would still be marked as it is today. Nothing needs to change.

But with that separate property being available per stage, visualizers like blue ocean could implement a configuration: "showStageLegacyStatus" or "showStageIndependentStatus", defined in the options for a pipeline. If the user chooses the latter, visualizers like blue ocean could make their own choice to paint strictly by the stageIndependentStatus.

I'v created a shared library, which contains an API to get the failed stages:

https://github.com/roel0/jenkins-util-scripts

You can then easily print out that data, or email it , or post it on slack or ...

Example usage:

try {
    super_stage("stage1") {
    }
    super_stage("stage2") {
        error "oei"
    }
} catch(e) {
}

println "Failed stages: ${get_failed_stages().join(',')}"

Some changes have recently been released to address this issue for Blue Ocean. All of the changes require Jenkins 2.138.4 or newer. The main fixes are in Pipeline: API Plugin version 2.34 and Pipeline: Graph Analysis Plugin version 1.10, but in most cases you will need to change your Pipelines to be able to take advantage of the fixes:

• If you are currently setting the build result to unstable using currentBuild.result = 'UNSTABLE', you need to update to Pipeline: Basic Steps Plugin version 2.16 and start using the unstable step instead. See the changelog here for details. You may also be interested in the new warnError step.
  • In case you are curious why we can't make currentBuild.result = 'UNSTABLE' work as-is, the problem is that currentBuild is implemented as a GlobalVariable rather than a regular Pipeline Step, so there is no way to associate the setting of the build result with the current Pipeline stage, especially if a parallel step is being executed.
• If the build result is being set to unstable by the junit step from the JUnit Plugin, then update the JUnit Plugin to version 1.28 and the visualization should be fixed.
• If the build result is being set to unstable by the publishIssues or recordIssues steps in the Warnings NG Plugin, then keep an eye on this in-progress PR which will update those steps to use a new API that will fix the visualization.
• If the build result is being set to unstable by a Pipeline step in some other plugin, then file a new issue with the component set to that plugin, and ask the maintainer to start using the new WarningAction API from Pipeline: Basic Steps Plugin version 2.16 wherever they change the build result.

In case of problems, the new behavior can be disabled by setting the system property org.jenkinsci.plugins.workflow.pipelinegraphanalysis.StatusAndTiming.DISABLE_WARNING_ACTION_LOOKUP to "true", or by setting the static variable of the same name to true via the Jenkins script console.

Note that the Blue Ocean steps view will not yet display the status of individual steps using the new API correctly, keep an eye on this in-progress PR which will fix that issue. Once that PR is merged and released, I plan to mark this issue as closed.

The visualization provided by Pipeline: Stage View Plugin has not been integrated with the new API added in Pipeline: API 2.34, and so that visualization is not currently affected by these changes.

Thank you for making progress on this issue.

I confirm it works, using it as follow:

steps { 
    script {
         def r = sh(returnStatus: true, script: "my_script");
         if (r != 0) {
             unstable(message: "Failed: ${STAGE_NAME} => return UNSTABLE status")
         }
    }
} 

 edit: ok it runs smoothly

I tried the new functionality dnusbaum mentioned above, but using anything else than the default values in catchError e.g. catchError(buildResult: 'SUCCESS') results in:

 
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 11: Expecting "class hudson.model.Result" for parameter "buildResult" but got "SUCCESS" of type class java.lang.String instead @ line 11, column 41.
catchError(buildResult: 'SUCCESS')

Jenkins version: 2.164.3

Pipeline Basic Steps: 2.16

(blue ocean is not installed)

It's Working!!!!

 Thanks Devin

Can one please post a screenshot how this looks now on traditional view and on BlueOcean?

Hm, while it seems to work fine for junit steps like in the screenshot it does not visualize the unstable stage when using code like hugo_l (in a replayed scripted pipeline):
How comes!?

twalter: using the fully qualified names like hudson.model.Result.FAILURE works for me.

twalter I just tested your code and it works ok for me in some basic tests. Please open a new issue for your problem, assign it to me, add any additional information or context that might be useful and include a list of all plugins you have installed and their versions. Thanks!

reinholdfuereder Can you please open a new issue, including a minimal Pipeline that reproduces the issue and assign the issue to me? Thanks!

Also I was not explicit in my original post that these fixes do not currently affect Stage View at all, they only affect Blue Ocean, so I will update my post to make that clear.

dnusbaum Please see JENKINS-57579

The visualization provided by Pipeline: Stage View Plugin has not been integrated with the new API added in Pipeline: API 2.34, and so that visualization is not currently affected by these changes.

I looked but could not find one; is the intention to have a separate ticket for the Stage View plugin integration?

I just published a blog post with an overview of this issue and how to take advantage of the recent changes in your Pipeline to improve the way it is visualized in Blue Ocean: https://jenkins.io/blog/2019/07/05/jenkins-pipeline-stage-result-visualization-improvements/. 

cburroughs Yes, feel free to open a new issue for Stage View integration and link it to this one.

Given the core changes have been released, I am closing this issue. As noted in my blog post, if you are still having problems with a Pipeline step from specific please file a new issue with the component set to that plugin (after checking for duplicates), clearly describing which step has the problem and under what circumstances it occurs, and link to the developers section of the post as a reference for how the maintainer might be able to address the problem.
 

Just created the issue for Pipeline: Stage View Plugin -> https://issues.jenkins-ci.org/browse/JENKINS-58783

Thanks for the hard work. I've just recently started to use Jenkins but on a older version. Then came along this issue while I was googling about the stage result issue. Glad there's already a solution in the recent version!  Appreciate it!

Was this more than just a visualisation problem or was this actually making stage B run it's post->unstable steps if stage A was UNSTABLE and set the overall build status to UNSTABLE?  Because that would be very bad/wrong.

Devin's blog post explains this issue and the fix well brianjmurrell

The fix doesn't make B Stage run, or not run - it still leaves it down to the implementor to choose if they want B to run or not

I have read Devin's blog post.  It does not answer my question AFAIU.  But you might think it could because you misunderstood my question.

My question was not about whole stages running or not running.  Both stages A and B are running.  B was SUCCESSFUL and A was UNSTABLE.  What I am seeing however is that B's post->unstable is being run even though B was successful.

I am trying to determine if that mis-behaviour is a component of this bug or if this is something different/new.

brianjmurrell Declarative post conditions look at the overall build result, so if a step like unstable or junit marks the build as unstable in stage A and the Pipeline continues executing, unstable post conditions in later stages such as B will execute (unless the build result becomes worse than unstable, e.g. failure). As far as I know Declarative has always worked this way. Before the fix for this issue, both stage A and stage B would have been shown as being unstable in Blue Ocean, but after the fix for this issue, only stage A is shown as unstable (the overall build result was shown as unstable both before and after the fix).

dnusbaum Whoa!  Are you serious (and sure of what you are saying)?  It seems very un-intuitive, at best, that the status of any individual given stage should be able to alter the status of any other stage.  I would absolutely agree that the outer pipeline {} block's post should be executed per the overall build result, but most certainly not any individual stage.

I suppose there must be some, can't imagine a use-case where that is useful.  On the contrary, if one has a parallel block of test stages, why would I want the unstable post block of test stage A to run when test stage A passed, but test stage B failed?

Ultimately, I suppose this should move to a new ticket.  I will open one.  But I probably need to mock up a minimal reproducer before I do.

dnusbaum Can you please respond to the previous comment?  Is it really the expected behaviour that all stages in a parallel block execute their post > failure blocks when even just 1 other stage in the parallel group fails?  The result being that stages that actually were successful don't actually process their own post > successful block but instead process their own post -> failure block?

Quite frankly, this behaviour is hideous.  In my case, a stage is not actually even failing.  In a block of 3 parallel stages if just one of them invokes a catchError(stageResult: 'UNSTABLE', buildResult: 'SUCCESS'), the remaining stages take their post > failure exit path, even though they are SUCCESSFUL, and even though the stage that invoked the catchError() takes it's own post > successful path!

I cannot imagine how this is not a bug.  This makes catchError(stageResult: 'UNSTABLE', buildResult: 'SUCCESS') quite useless in fact if it can bomb the exit of other stages.

JENKINS-57801 has been filed about declarative Pipeline stage post conditions being affected by the result of the whole pipeline.