Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18696

Not all test results are being reported

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • junit-plugin
    • CentOS

      Our build process produces multiple directories of JUnit result XML files. The workspace structure looks like this:

         build/
      integration-a/
         test-results/
      integration-b/
         test-results/
      integration-c/
         test-results/
      integration-d/
         test-results/

      The project is configured to collect the test results using the following file path:

      build/integration-*/test-results/*.xml

      I have verified looking in the workspace after the build that all the XMl files are there and that all the tests have run.

      The problem is that Jenkins is reporting slightly different counts of unit tests every build. I've looked back over older builds and this has been happening for months (could be even longer, but I just switched servers and lost my older history). For example, the last few build test counts have looked like this (the number of tests should not have changed at all):

      #23 18,875 tests (+210)
#22 18,665 tests (±0)
#21 18,665 tests (+204)
#20 18,461 tests (-401)
#19 18,862 tests (+186)
#18 18,676 tests (-147)
#17 18,823 tests (-39)

      When I review the test results in Jenkins, I find every test that it doesn't list. When I look at the workspace, I can find the XML file and there are also HTML files (in build/integration-*/reports/tests/) that are produced and all the tests are there as well. It just seems Jenkins is dropping various test XML files on each build. I just can't see a pattern. The count seems different every time.

          [JENKINS-18696] Not all test results are being reported

          Mathieu Cantin added a comment -

          We have the same issue on windows server 2012R2 with jenkins 1.596.2 and JUnit plugin 1.5.

          Mathieu Cantin added a comment - We have the same issue on windows server 2012R2 with jenkins 1.596.2 and JUnit plugin 1.5.

          Aaron Humerickhouse added a comment - - edited

          Have the same issue. I was on 1.2-beta4 (default with Jenkins), just updated to 1.5 and am seeing the same issue.

          Files
          selenium-tests
          -results
          --results0.xml
          --results1.xml
          --results2.xml
          --results3.xml
          --results4.xml

          I've tried:
          **/results*.xml
          selenium-tests/results/results*.xml
          **/results/results.xml
          selenium-tests/results/results0.xml,selenium-tests/results/results1.xml,selenium-tests/results/results2.xml,selenium-tests/results/results3.xml,selenium-tests/results/results4.xml

          Four different patterns that should get me all of the files are failing to do so.

          Aaron Humerickhouse added a comment - - edited Have the same issue. I was on 1.2-beta4 (default with Jenkins), just updated to 1.5 and am seeing the same issue. Files selenium-tests -results --results0.xml --results1.xml --results2.xml --results3.xml --results4.xml I've tried: **/results*.xml selenium-tests/results/results*.xml **/results/results.xml selenium-tests/results/results0.xml,selenium-tests/results/results1.xml,selenium-tests/results/results2.xml,selenium-tests/results/results3.xml,selenium-tests/results/results4.xml Four different patterns that should get me all of the files are failing to do so.

          Kevin Neal added a comment -

          Usually the newer issue duplicates the older issue, but I found that one first and submitted a PR today. This issue has a PR open, but it's out of date.

          Kevin Neal added a comment - Usually the newer issue duplicates the older issue, but I found that one first and submitted a PR today. This issue has a PR open, but it's out of date.

            khneal Kevin Neal
            mayc Chuck May
            Votes:
            4 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: