-
Bug
-
Resolution: Not A Defect
-
Blocker
-
None
It seems like the "testsuite" element in my junit.xml file that Jenkins aggregated is not correct.
Somehow, the "Test Result" page I see in Jenkins shows me the correct numbers - good job with that! However, when I examine the junit.xml file, the errors, failures, skips, and tests elements do not match what I see in Jenkins.
I use Jenkins parallel jobs to run my tests because I have thousands of python unit tests that are run with py.test. I use py.test's junitxml output file which I later ingest into jenkins.
My projects has over 5,000 tests. In a recent run, the junit file suggests there were 0 failures and 1 skips which is wrong....
<?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="0" name="pytest" skips="1" tests="33"....
However, the Jenkins rendering engine was able to correctly deduce that there really are 11 failures at 525 skipped tests as you can see in my attached screenshot.
I think the bug here is that as each parallel shard is ingested by the jenkins master, that the errors, failures, skips, and tests count needs to be updated.
