Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4963

xunit plugin parses test logs and not test results of Boost.Test

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Hello,

      Boost.Test produces two different outputs :
      . the log is sent to the standard output during the tests run
      . the result report is sent to the standard error output at the end of the run
      (see http://www.boost.org/doc/libs/1_41_0/libs/test/doc/html/utf/user-guide/test-output.html )
      Each output can be formatted either to text or to XML (but not both).

      It is really convenient to keep the log in human readable format as it helps pinpoint where problems occur (including crashes).
      When a build fails, compilation and test logs are where a lot of the problems are immediately diagnosed.

      The xUnit plugin only accepts the XML log as input, which prevents it from being usable in our environment (and I suspect in many environments although I certainly don't have any proof of that ).

      It would be great to support both the log and/or the result report.
      It makes sense, after all the goal of the xUnit plugin is "to publish testing tools test result report", isn't it ?

      Here is what the test result report looks like :

      <TestResult>
      <TestSuite name="xeumeuleu_xerces2_test" result="failed" assertions_passed="248" assertions_failed="2" expected_failures="0" test_cases_passed="207" test_cases_failed="2" test_cases_skipped="0" test_cases_aborted="0">
      <TestCase name="read_attribute_from_root_level_throws_an_exception" result="passed" assertions_passed="1" assertions_failed="0" expected_failures="0"></TestCase>
      <TestCase name="read_unexisting_attribute_throws_an_exception" result="passed" assertions_passed="1" assertions_failed="0" expected_failures="0"></TestCase>
      ...

      Thanks !
      MAT.

        Attachments

          Activity

          Hide
          gbois Gregory Boissinot added a comment - - edited

          I'm afraid I don't understand your request.

          In my opinion:
          --report_format=XML --report_level=detailed produces the result report
          --log_level=all produces the log report with the result report
          --output_format=XML --log_level=all --report_level=no produces only the log report

          And with the third option "--output_format=XML --log_level=all", you have all the information you need.

          Please could you give me an example where these informations are in test results reports and they are not in the test logs report produced by "--output_format=XML --log_level=all"?

          Show
          gbois Gregory Boissinot added a comment - - edited I'm afraid I don't understand your request. In my opinion: --report_format=XML --report_level=detailed produces the result report --log_level=all produces the log report with the result report --output_format=XML --log_level=all --report_level=no produces only the log report And with the third option "--output_format=XML --log_level=all", you have all the information you need. Please could you give me an example where these informations are in test results reports and they are not in the test logs report produced by "--output_format=XML --log_level=all"?
          Hide
          mat007 mat007 added a comment -

          Thanks for looking into this !

          The problem is that I would like to keep the log report in text format and therefore cannot use --output_format=XML
          The result report however can be in XML format, but the xunit plugin only reads the log report.

          For instance here is what the log looks like in the hudson console log (it's run using ant):

          validation_test:
          [test] Running 26 test cases...
          [test] D:/hudson/jobs/mch/workspace/platform/vc80/trunk/src/tests/validation_test/ValidationTest.cpp(117): error in "T_M_C_3_stroll_around_with_a_civilian": Distance is bigger than 45 because the distance is 47.7901611

          BUILD FAILED

          If I were to switch the log report to XML (and redirect it to a file) then the log would look like :

          validation_test:

          BUILD FAILED

          I wouldn't want to have to open a file and read through some XML in order to understand why the tests failed.

          Show
          mat007 mat007 added a comment - Thanks for looking into this ! The problem is that I would like to keep the log report in text format and therefore cannot use --output_format=XML The result report however can be in XML format, but the xunit plugin only reads the log report. For instance here is what the log looks like in the hudson console log (it's run using ant): validation_test: [test] Running 26 test cases... [test] D:/hudson/jobs/mch/workspace/platform/vc80/trunk/src/tests/validation_test/ValidationTest.cpp(117): error in "T_M_C_3_stroll_around_with_a_civilian": Distance is bigger than 45 because the distance is 47.7901611 BUILD FAILED If I were to switch the log report to XML (and redirect it to a file) then the log would look like : validation_test: BUILD FAILED I wouldn't want to have to open a file and read through some XML in order to understand why the tests failed.
          Hide
          gbois Gregory Boissinot added a comment -

          I'm afraid this issue is not possible.
          For an Hudson integration, you must keep test results in XML and you must use Hudson test results browsing features.

          Show
          gbois Gregory Boissinot added a comment - I'm afraid this issue is not possible. For an Hudson integration, you must keep test results in XML and you must use Hudson test results browsing features.

            People

            Assignee:
            gbois Gregory Boissinot
            Reporter:
            mat007 mat007
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: