-
Bug
-
Resolution: Fixed
-
Minor
node('master') { stage('1') { step([$class: 'XUnitBuilder', thresholds: [[$class: 'FailedThreshold', failureThreshold: "1"]], tools: [[$class: 'JUnitType', pattern: "tmp.xml"]]]) } stage('2') { step([$class: 'XUnitBuilder', thresholds: [[$class: 'FailedThreshold', failureThreshold: "0"]], tools: [[$class: 'JUnitType', pattern: "tmp2.xml"]]]) } }
tmp.xml contains one failure and tmp2.xml contains no failures.
The result is as followed:
[test_pipeline] Running shell script [Pipeline] step [xUnit] [INFO] - Starting to record. [xUnit] [INFO] - Processing JUnit [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern 'tmp.xml' relative to '/var/lib/jenkins/workspace/test_pipeline' for the testing framework 'JUnit'. [xUnit] [INFO] - Check 'Failed Tests' threshold. [xUnit] [INFO] - Setting the build status to SUCCESS [xUnit] [INFO] - Stopping recording. [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (2) [Pipeline] sh [test_pipeline] Running shell script [Pipeline] step [xUnit] [INFO] - Starting to record. [xUnit] [INFO] - Processing JUnit [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern 'tmp2.xml' relative to '/var/lib/jenkins/workspace/test_pipeline' for the testing framework 'JUnit'. [xUnit] [INFO] - Check 'Failed Tests' threshold. [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value. [xUnit] [INFO] - Setting the build status to FAILURE [xUnit] [INFO] - Stopping recording.
Xunit keeps track of previous recorded failures, while it, in my opinion, should not
- is duplicated by
-
-
- Resolved
-
[JENKINS-47194] xUnit should not consider report generated by other xUnit step
Code changed in jenkins
User: Nikolas Falco
Path:
src/main/java/org/jenkinsci/plugins/xunit/XUnitProcessor.java
src/main/java/org/jenkinsci/plugins/xunit/service/XUnitTransformer.java
src/test/java/org/jenkinsci/plugins/xunit/XUnitPublisherTest.java
src/test/java/org/jenkinsci/plugins/xunit/XUnitWorkflowTest.java
src/test/java/org/jenkinsci/plugins/xunit/service/TempWorkspace.java
src/test/java/org/jenkinsci/plugins/xunit/service/XUnitTransformerTest.java
src/test/java/org/jenkinsci/plugins/xunit/service/XUnitValidationServiceTest.java
src/test/resources/org/jenkinsci/plugins/xunit/XUnitPublisherTest/workspace/JENKINS-47194/input.xml
src/test/resources/org/jenkinsci/plugins/xunit/config.xml
http://jenkins-ci.org/commit/xunit-plugin/10d52804cbf03c855fc0b9208ff6d8a62cf0efa0
Log:
JENKINS-47194 xUnit should not consider report generated by other xUnit step
XUnit build step or publisher use an instance of processor than generates output report to a shared common folder. This cause on latest step thresholds consider also reports produced by previous steps. Processor now generate an identifier that permits to generate reports into an unique output folder.
*NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/
Functionality will be removed from GitHub.com on January 31st, 2019.
I'm still experiencing this problem with xUnit 2.0.0 on jenkins 2.126. I just posted to jenkinsci-users mailing list: https://groups.google.com/d/msg/jenkinsci-users/Wk2QfWdsRFo/8pNpN3jDBAAJ
Does the fix in 2.0.0 work for anyone else?
This bug is not still fixed completely. Generated report files does not interfere with ones generated by other but thresholds are calculated against the global TestResult object in the TestResultAction for the current job.
Each step must generates single TestResult and use it for threasholds, but we need hack the TestResult object to use merge method.
nfalco ok good to know, thanks for the info!
Also when testing a bit more I found that the same problem occurs if one stage uses xUnit and the other stage uses the basic junit() step. Will your fix work only if all test result analysis uses xUnit?
The threshold will be always applied against the sum of the tools you had specify in the step, not anymore against the global sum of all gathered tests in the job.
JENKINS-27395may be relevant here.