-
Bug
-
Resolution: Unresolved
-
Critical
-
None
The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently ignored.
This happens with the jUnit publisher and xUnit. It even happens if only the timestamps are equal, which is a nasty bug since timestamps shouldn't be used to uniquely identify a testsuite when they have different names. It looks like Jenkins publishes the first suite and ignores any other XML files with the same timestamp. If the timestamps are empty, they are all reported. Reproduced with Jenkins 1.595 with junit 1.2-beta4, and also reproduced with the latest Jenkins 1.627 and junit 1.8.
See the two attached test_result_*.xml to reproduce: Archive them via junit and only testFoo is reported. If you change the timestamp attribute of either, they both are reported.
[JENKINS-28490] jUnit discarding <testsuite> tags when certain attributes are equal
Kris Reeves
created issue -
Kris Reeves
made changes -
| Environment |
New:
CentOS release 6.6 (Final) Linux jenkins 2.6.32-504.16.2.el6.x86_64 #1 SMP Wed Apr 22 06:48:29 UTC 2015 x86 OpenJDK Runtime Environment (rhel-2.5.5.1.el6_6-x86_64 u79-b14) OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode) Jenkins 1.611 xUnit plugin 1.96 jUnit plugin 1.6 |
Kris Reeves
made changes -
| Description | Original: I have a project set to publish test results using the xUnit plugin, jUnit format. After noticing some irregularities in the graph, I found that apparently the "junitResult.xml" file that is being created kind of arbitrarily doesn't have one of the test suites in the input xml file. Replacing the timestamps with static text, a diff between the input files comes up clean; a diff between the output files reveals that it appears to be the same section that is getting excluded. I've attached the input and output files below. The section being excluded sometimes is the section at line 69. |
New:
I have a project set to publish test results using the xUnit plugin, jUnit format. After noticing some irregularities in the graph, I found that apparently the "junitResult.xml" file that is being created kind of arbitrarily doesn't have one of the test suites in the input xml file. Replacing the timestamps with static text, a diff between the input files comes up clean; a diff between the output files reveals that it appears to be the same section that is getting excluded. I've attached the input and output files below. The section being excluded sometimes is the section at line 69. Update: The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently(?) ignored. |
| Summary | Original: jUnit (?) randomly not including test suites | New: jUnit (?) discarding <testsuite> tags when certain attributes are equal |
mcrooney
made changes -
| Attachment | New: test_result_9114558122204227176.xml [ 30604 ] | |
| Attachment | New: test_result_1012678300206172099.xml [ 30605 ] |
mcrooney
made changes -
| Summary | Original: jUnit (?) discarding <testsuite> tags when certain attributes are equal | New: jUnit discarding <testsuite> tags when certain attributes are equal |
mcrooney
made changes -
| Priority | Original: Minor [ 4 ] | New: Critical [ 2 ] |
mcrooney
made changes -
| Description |
Original:
I have a project set to publish test results using the xUnit plugin, jUnit format. After noticing some irregularities in the graph, I found that apparently the "junitResult.xml" file that is being created kind of arbitrarily doesn't have one of the test suites in the input xml file. Replacing the timestamps with static text, a diff between the input files comes up clean; a diff between the output files reveals that it appears to be the same section that is getting excluded. I've attached the input and output files below. The section being excluded sometimes is the section at line 69. Update: The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently(?) ignored. |
New: The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently ignored. |
mcrooney
made changes -
| Description | Original: The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently ignored. |
New:
The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently ignored. This happens with the jUnit publisher and xUnit. It even happens if only the timestamps are equal, which is a nasty bug since timestamps shouldn't be used to uniquely identify a testsuite when they have different names. It looks like Jenkins publishes the first suite and ignores any other XML files with the same timestamp. If the timestamps are empty, they are all reported. Reproduced with Jenkins 1.595 with junit 1.2-beta4, and also reproduced with the latest Jenkins 1.627 and junit 1.8. |
mcrooney
made changes -
| Description |
Original:
The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently ignored. This happens with the jUnit publisher and xUnit. It even happens if only the timestamps are equal, which is a nasty bug since timestamps shouldn't be used to uniquely identify a testsuite when they have different names. It looks like Jenkins publishes the first suite and ignores any other XML files with the same timestamp. If the timestamps are empty, they are all reported. Reproduced with Jenkins 1.595 with junit 1.2-beta4, and also reproduced with the latest Jenkins 1.627 and junit 1.8. |
New:
The cause of this problem appears to be the `name` and `timestamp` attributes of a `<testsuite>` tag being equal. When processing the report file, if a `<testsuite>` tag is encountered where these attributes are the same as one that was already seen, the new tag is silently ignored. This happens with the jUnit publisher and xUnit. It even happens if only the timestamps are equal, which is a nasty bug since timestamps shouldn't be used to uniquely identify a testsuite when they have different names. It looks like Jenkins publishes the first suite and ignores any other XML files with the same timestamp. If the timestamps are empty, they are all reported. Reproduced with Jenkins 1.595 with junit 1.2-beta4, and also reproduced with the latest Jenkins 1.627 and junit 1.8. See the two attached test_result_*.xml to reproduce: Archive them via junit and only testFoo is reported. If you change the timestamp attribute of either, they both are reported. |
| Workflow | Original: JNJira [ 163330 ] | New: JNJira + In-Review [ 181185 ] |
| Assignee | Original: Gregory Boissinot [ gbois ] |