-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
Jenkins 2.150.3, perfpublisher 8.08, many others
Since upgrading from 8.06 to 8.08 (which fixed some html issues for us), we have noticed that builds are no longer marked unstable when there are test errors.
It does not matter whether we set the parameter "threshold" (or "unstableThreshold") to blank or zero '0'.
To reproduce, make a pipeline with this configuration and run it:
pipeline {
agent any
stages {
stage('perf') {
steps {
writeFile file: 'result.xml', text: '''
<report categ="Automated test" name="Exhibit A">
<test executed="yes" name="A"><result><success unstable="false" passed="no" state="0" /><errorlog>FAIL</errorlog><executiontime isRelevant="true" mesure="1.000" unit="s" /><log>BAD</log></result></test>
<test executed="yes" name="B"><result><success unstable="false" passed="yes" state="100" /><errorlog></errorlog><executiontime isRelevant="true" mesure="1.000" unit="s" /><log>GOOD</log></result></test>
</report>
'''
perfpublisher name: 'result.xml', threshold: '0', unstableThreshold: '0', healthy: '', unhealthy: '', metrics: '', parseAllMetrics: false
}
}
}
}
Expected result: Build is unstable (yellow), because there's 1 test failure.
Actual result: Build is stable (blue/green).
The plugin does correctly show that there's 1 success and 1 failure, but does not set status correctly.
Here's an excerpt of the log output from the build:
[PerfPublisher] Compute global statistics... [PerfPublisher] [--------------------------------------------------] [PerfPublisher] Number of parsed files : 1 [PerfPublisher] Number of reports : 1 [PerfPublisher] Number of test : 2 [PerfPublisher] Number of executed test : 2 [PerfPublisher] Number of not executed test : 0 [PerfPublisher] Number of passed test : 1 [PerfPublisher] Number of failed test : 1 [PerfPublisher] --------------------------------------------------- [PerfPublisher] Category : Automated test [PerfPublisher] - Number of test : 2 [PerfPublisher] A : null [PerfPublisher] B : null [PerfPublisher] --------------------------------------------------- [PerfPublisher] Analysis : [PerfPublisher] ---------------------------------------------------