Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57615

xUnit fails to parse PHPUnit output when a skipped test also has system output

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I'm using the latest version of the xUnit plugin (2.3.3) with the PHPUnit-3.x parser. It fails on a few tests because it can't parse the xml file, and it gives a somewhat misleading error:

      WARNING: The file 'myresult.xml' is an invalid file.
      WARNING: At line 7261 of file:myresult.xml:cvc-complex-type.2.4.d: Invalid content was found starting with element 'system-out'. No child element is expected at this point.
      FATAL: The result file 'myresult.xml' for the metric 'PHPUnit' is not valid. The result file has been skipped.

      This seems to be happening because the test was skipped, but still had system output. Here's an example:

      <testcase name="testMyTest" class="MyTestUnitTest" file="myUnitTest.php" line="320" assertions="157" time="45.286155">
          <skipped />
          <system-out>Some debug info...</system-out>
      </testcase>

      In this case there's a bunch of setup happening in the base class before the test even runs (which will ultimately be skipped), which is echoing out debug information. This system output is useful for debugging, and inconsequential to the result of the tests. I can alter my tests to not output anything, but parsing shouldn't fail here, or should at least produce a meaningful error message.

        Attachments

          Issue Links

            Activity

            Hide
            captain_hypertext Captain Hypertext added a comment -

            Yes, I did think it was weird how things were fixed when I just switched the parser away from JUnit to PHPUnit, but hey, it works for everything except these cases.  The log types available for my version of PHPUnit is: junit, teamcity, testdox-html, testdox-text, testdox-xml, and reverse-list.

            Show
            captain_hypertext Captain Hypertext added a comment - Yes, I did think it was weird how things were fixed when I just switched the parser away from JUnit to PHPUnit, but hey, it works for everything except these cases.  The log types available for my version of PHPUnit is: junit, teamcity, testdox-html, testdox-text, testdox-xml, and reverse-list.
            Hide
            nfalco Nikolas Falco added a comment -

            The case is the same of other similar issue. The point is that PHPUnit does not produce a supported JUnit format. They declare to produce a format junit task like but is not true. The reason of issue https://github.com/sebastianbergmann/phpunit/issues/3162 where they would standardize the format in output.

            In the Wiki is explained how to setup "In case you are using a non java tools that declares to produce a JUnit compatible report" (search the string between "")

            Show
            nfalco Nikolas Falco added a comment - The case is the same of other similar issue. The point is that PHPUnit does not produce a supported JUnit format. They declare to produce a format junit task like but is not true. The reason of issue https://github.com/sebastianbergmann/phpunit/issues/3162 where they would standardize the format in output. In the Wiki is explained how to setup "In case you are using a non java tools that declares to produce a JUnit compatible report" (search the string between "")
            Hide
            captain_hypertext Captain Hypertext added a comment -

            Yes, but this does not address my issue.  My issue is with the PHPUnit-3.x parser, which works fine except for this bug.  I am not using the JUnit parser.

            Show
            captain_hypertext Captain Hypertext added a comment - Yes, but this does not address my issue.  My issue is with the PHPUnit-3.x parser, which works fine except for this bug.  I am not using the JUnit parser.
            Hide
            nfalco Nikolas Falco added a comment -

            phpunit --log-junit produces a JUnit report.

            Show
            nfalco Nikolas Falco added a comment - phpunit --log-junit produces a JUnit report.
            Hide
            captain_hypertext Captain Hypertext added a comment -

            Yes, I know.  Please explain how the PHPUnit-3.x parser works better than the JUnit parser for parsing JUnit logs, or at least refer me to the author or proper documentation.  This is definitely not a phpUnit issue, this bug is an issue with the PHPUnit parser in xUnit.

            Show
            captain_hypertext Captain Hypertext added a comment - Yes, I know.  Please explain how the PHPUnit-3.x parser works better than the JUnit parser for parsing JUnit logs, or at least refer me to the author or proper documentation.  This is definitely not a phpUnit issue, this bug is an issue with the PHPUnit parser in xUnit.

              People

              Assignee:
              nfalco Nikolas Falco
              Reporter:
              captain_hypertext Captain Hypertext
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: