-
Bug
-
Resolution: Not A Defect
-
Minor
-
None
I'm using the latest version of the xUnit plugin (2.3.3) with the PHPUnit-3.x parser. It fails on a few tests because it can't parse the xml file, and it gives a somewhat misleading error:
WARNING: The file 'myresult.xml' is an invalid file. WARNING: At line 7261 of file:myresult.xml:cvc-complex-type.2.4.d: Invalid content was found starting with element 'system-out'. No child element is expected at this point. FATAL: The result file 'myresult.xml' for the metric 'PHPUnit' is not valid. The result file has been skipped.
This seems to be happening because the test was skipped, but still had system output. Here's an example:
<testcase name="testMyTest" class="MyTestUnitTest" file="myUnitTest.php" line="320" assertions="157" time="45.286155">
<skipped />
<system-out>Some debug info...</system-out>
</testcase>
In this case there's a bunch of setup happening in the base class before the test even runs (which will ultimately be skipped), which is echoing out debug information. This system output is useful for debugging, and inconsequential to the result of the tests. I can alter my tests to not output anything, but parsing shouldn't fail here, or should at least produce a meaningful error message.
- is related to
-
JENKINS-52392 Support custom junit format that does not respect any precise schema
-
- Resolved
-
[JENKINS-57615] xUnit fails to parse PHPUnit output when a skipped test also has system output
Captain Hypertext
created issue -
Captain Hypertext
made changes -
| Description |
Original:
I'm using the latest version of the xUnit plugin (2.3.3) with the PHPUnit-3.x parser. It fails on a few tests because it can't parse the xml file, and it gives a somewhat misleading error: {noformat} WARNING: The file 'myresult.xml' is an invalid file. WARNING: At line 7261 of file:myresult.xml:cvc-complex-type.2.4.d: Invalid content was found starting with element 'system-out'. No child element is expected at this point. FATAL: The result file 'myresult.xml' for the metric 'PHPUnit' is not valid. The result file has been skipped.{noformat} This seems to be happening because the test was skipped, but still had system output. Here's an example: {noformat} <testcase name="testMyTest" class="MyTestUnitTest" file="myUnitTest.php" line="320" assertions="157" time="45.286155"> <skipped /> <system-out>Some debug info...</system-out> </testcase>{noformat} In this case there's a bunch of setup happening in the base class before the test even runs (which will ultimately be skipped), which is echoing out debug information. This system output is useful for debugging, and inconsequential to the result of the tests. I can alter my tests to not output anything, but parsing shouldn't fail here, or should at least produce a meaningful error message. |
New:
I'm using the latest version of the xUnit plugin (2.3.3) with the PHPUnit-3.x parser. It fails on a few tests because it can't parse the xml file, and it gives a somewhat misleading error: {noformat} WARNING: The file 'myresult.xml' is an invalid file. WARNING: At line 7261 of file:myresult.xml:cvc-complex-type.2.4.d: Invalid content was found starting with element 'system-out'. No child element is expected at this point. FATAL: The result file 'myresult.xml' for the metric 'PHPUnit' is not valid. The result file has been skipped.{noformat} This seems to be happening because the test was skipped, but still had system output. Here's an example: {noformat} <testcase name="testMyTest" class="MyTestUnitTest" file="myUnitTest.php" line="320" assertions="157" time="45.286155"> <skipped /> <system-out>Some debug info...</system-out> </testcase>{noformat} In this case there's a bunch of setup happening in the base class before the test even runs (which will ultimately be skipped), which is echoing out debug information. This system output is useful for debugging, and inconsequential to the result of the tests. I can alter my tests to not output anything, but parsing shouldn't fail here, or should at least produce a meaningful error message. |
Captain Hypertext
made changes -
| Attachment | New: config.xml [ 47473 ] |
Captain Hypertext
added a comment - nfalco - Just uploaded my job's config.xml I see that in the config.xml you are running "phpunit --log-junit .." that by documentation here it produce a "JUnit Ant like". You had configured PHPUnit as report type that means it expects to parse a different format.
Change PHPUnit 4.x -> JUnit and hope that the JUnit produced is a really Ant task format like the documentation describe (https://github.com/sebastianbergmann/phpunit/issues/3162).
| Resolution | New: Not A Defect [ 7 ] | |
| Status | Original: Open [ 1 ] | New: Closed [ 6 ] |
Captain Hypertext
added a comment - I had to switch from JUnit because it stopped parsing after we upgraded. I think it's related to JENKINS-51919, it kept failing to parse because of the attributes PHPUnit includes such as "assertions", along with several more. So I switched to PHPUnit-3.x (PHPUnit-4.x was not listed as an option, and my version of PHPUnit is 6, so I'm confused about that...). I'll see if I can get one of those report files for you.
Captain Hypertext
added a comment - I had to switch from JUnit because it stopped parsing after we upgraded. I think it's related to JENKINS-51919 , it kept failing to parse because of the attributes PHPUnit includes such as "assertions", along with several more. So I switched to PHPUnit-3.x (PHPUnit-4.x was not listed as an option, and my version of PHPUnit is 6, so I'm confused about that...). I'll see if I can get one of those report files for you. Captain Hypertext
added a comment - Yes, I did think it was weird how things were fixed when I just switched the parser away from JUnit to PHPUnit, but hey, it works for everything except these cases. The log types available for my version of PHPUnit is: junit, teamcity, testdox-html, testdox-text, testdox-xml, and reverse-list.
Captain Hypertext
added a comment - Yes, I did think it was weird how things were fixed when I just switched the parser away from JUnit to PHPUnit, but hey, it works for everything except these cases. The log types available for my version of PHPUnit is: junit, teamcity, testdox-html, testdox-text, testdox-xml, and reverse-list.
Could you post your job configuration? I would know your xunit step configuration.