-
Bug
-
Resolution: Unresolved
-
Minor
-
None
While developing a matrix test for https://github.com/networkupstools/nut/pull/947 I found that it is inconvenient to have the whole crossplatform test suite fail while we are cleaning it up vs. new linters.
So I wrapped the tests into `catchError() {...}` closures to still have a verdict on every individual test. And saw that everything came up green... except the overall result is red (as anticipated) – see nut-jenkinsfile-869-screenshot-catchError-redAllGreens.png:
I began looking into individual Blue Ocean test results and found many failures (that were expected). In fact, not only the BO circles are green for those failed tests, but also the hovering tooltips are green despite the test step log being correctly failed and red – see nut-jenkinsfile-869-screenshot-catchError-cropped.png:
Finally, changing the code block wrapping into `warnError() {...}` did highlight the individual tests (icons and tooltips) as unstable, and marked the overall build unstable – see nut-jenkinsfile-869-screenshot-warnError-unstable.png for overall view:
...and nut-jenkinsfile-869-screenshot-warnError.png for the failed test's red log:
UPDATED: Original posting had a typo, naming "checkError" instead of "catchError" - double-checked that the latter was indeed the step requested in that Jenkinsfile.