Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61766

JUnit report misses test cases from duplicate testsuites

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • junit-plugin
    • Jenkins ver. 2.204.6
      JUnit Plugin (junit): 1.28
      OS: Ubuntu Linux 18.04

      When the same testsuite appears in multiple files with the same timestamp, only the first testsuite is evaluated, any testcases in following files are silently ignored.

       

      I attached two test report XML files with typical output of our build runs, yet simplified as much as possible. As you can see, both files contain a different test case of the same testsuite "TestFoo". If the test execution is very fast it can happen that both the testsuites have the same timestamp, in which case every testcase but the first is not included in the report, as you can see on the attached screenshot.

       

      The expected behaviour is, that all the testcases are included in the report and the stage fails if any of the testcases failed.

       

      I consider this a major bug as it allows failed tests to be ignored and in the consequence broken software can be reported as "successfull" when it isn't.

       

        1. JUnit Report.png
          JUnit Report.png
          7 kB
        2. test_1.xml
          0.4 kB
        3. test_2.xml
          0.4 kB

            Unassigned Unassigned
            exomo Kai Bublitz
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: