-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
-
3.0.6
We use xunit in a parallel pipeline to analyze test report files.
When there are multiple test failures, xunit only marks the first calling stage as unstable. All other stages are marked as passed.
Here is an stage overview of a MWE illustrating the problem:
All three stages perform the xunit analysis on the same test report file which contains one failing test. The xunit call is like so:
xunit thresholds: [failed(unstableThreshold: '0')], tools: [GoogleTest(deleteOutputFiles: true, failIfNotNew: false, pattern: "*.xml", skipNoTestFiles: false, stopProcessingIfError: true)]
The only difference between the stages is a delay before calling xunit which has been swapped between the three stages for each run.
I would expect all three stages to be marked unstable, like so:
We use xunit version 3.0.5, and Jenkins Core 2.319.3
[JENKINS-68061] On parallel execution stages are not marked according to threshold values
O B
created issue -
O B
made changes -
| Attachment | New: unstable_test.xml [ 57482 ] | |
| Attachment | New: unstable_test.groovy [ 57483 ] |
O B
made changes -
| Description |
Original:
We use xunit in a parallel pipeline to analyze test report files.
When there are multiple test failures, xunit only marks the first calling stage as unstable. All other stages are marked as passed. Here is an stage overview of a MWE illustrating the problem: !image-2022-03-17-11-38-41-869.png! All three stages perform the xunit analysis on the same test report file which contains one failing test. The xunit call is like so: {code:groovy} xunit thresholds: [failed(unstableThreshold: '0')], tools: [GoogleTest(deleteOutputFiles: true, failIfNotNew: false, pattern: "*.xml", skipNoTestFiles: false, stopProcessingIfError: true)] {code} The only difference between the stages is a delay before calling xunit which has been swapped between the three stages for each run. I would expect all three stages to be marked unstable, like so: !image-2022-03-17-11-45-06-156.png|thumbnail! |
New:
We use xunit in a parallel pipeline to analyze test report files.
When there are multiple test failures, xunit only marks the first calling stage as unstable. All other stages are marked as passed. Here is an stage overview of a MWE illustrating the problem: !image-2022-03-17-11-38-41-869.png! All three stages perform the xunit analysis on the same test report file which contains one failing test. The xunit call is like so: {code:groovy} xunit thresholds: [failed(unstableThreshold: '0')], tools: [GoogleTest(deleteOutputFiles: true, failIfNotNew: false, pattern: "*.xml", skipNoTestFiles: false, stopProcessingIfError: true)] {code} The only difference between the stages is a delay before calling xunit which has been swapped between the three stages for each run. I would expect all three stages to be marked unstable, like so: !image-2022-03-17-11-45-06-156.png|thumbnail! ---- We use xunit version 3.0.5, and Jenkins Core 2.319.3 |
| Issue Type | Original: Improvement [ 4 ] | New: Bug [ 1 ] |
| Status | Original: Open [ 1 ] | New: In Progress [ 3 ] |
| Summary | Original: Set all stages according to threshold values | New: On parallel execution stages are not marked according to threshold values |
| Resolution | New: Fixed [ 1 ] | |
| Status | Original: In Progress [ 3 ] | New: Fixed but Unreleased [ 10203 ] |
| Released As | New: 3.0.6 | |
| Status | Original: Fixed but Unreleased [ 10203 ] | New: Resolved [ 5 ] |
O B
made changes -
| Status | Original: Resolved [ 5 ] | New: Fixed but Unreleased [ 10203 ] |
| Status | Original: Fixed but Unreleased [ 10203 ] | New: Resolved [ 5 ] |