• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • maven-plugin
    • Platform: All, OS: All

      Using the Maven job type, if the surefire results are run twice (eg clean
      package site) the test results are counted twice.

      If we use the same pom for a freestyle project, the results are shown correctly.

          [JENKINS-2068] Junit test showing twice

          Chris Graham created issue -

          breadfan_de added a comment -

          Test cases are also counted twice for any test coverage tool which executes
          tests with instrumented class files

          breadfan_de added a comment - Test cases are also counted twice for any test coverage tool which executes tests with instrumented class files

          retronym added a comment -

          We have two surefire executions – one for unit tests, and one for integration
          tests. This also reports each test twice.

          retronym added a comment - We have two surefire executions – one for unit tests, and one for integration tests. This also reports each test twice.

          Tim Pesce added a comment -

          We're seeing the same problem, and I have an idea about what might be going on.

          We have a setup that is using surefire for both unit and integration tests like retronym. In our case we
          have 3 unit tests and 2 integration tests. When the unit test phase completes, Hudson seems to go look
          for test results:

          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

          [HUDSON] Recording test results

          Then, when the integration test phase completes Hudson seems to go look for test results again:

          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

          [HUDSON] Recording test results

          I think the results of the unit tests are being collected twice. In our total test count we see unit tests * 2
          + integration tests. The test result drill downs also show duplicate entries for the unit tests.

          Tim Pesce added a comment - We're seeing the same problem, and I have an idea about what might be going on. We have a setup that is using surefire for both unit and integration tests like retronym. In our case we have 3 unit tests and 2 integration tests. When the unit test phase completes, Hudson seems to go look for test results: Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 [HUDSON] Recording test results Then, when the integration test phase completes Hudson seems to go look for test results again: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 [HUDSON] Recording test results I think the results of the unit tests are being collected twice. In our total test count we see unit tests * 2 + integration tests. The test result drill downs also show duplicate entries for the unit tests.

          Aside from actually coding a fix, is there any other way I can help with it?
          It's really troublesome, since it makes the m2 project type much less useful.

          Robert Munteanu added a comment - Aside from actually coding a fix, is there any other way I can help with it? It's really troublesome, since it makes the m2 project type much less useful.
          Andrew Bayer made changes -
          Link New: This issue duplicates JENKINS-1557 [ JENKINS-1557 ]

          Andrew Bayer added a comment -

          Duplicate of JENKINS-1557.

          Andrew Bayer added a comment - Duplicate of JENKINS-1557 .
          Andrew Bayer made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

          John McNair added a comment -

          I think there are two separate issues here. JENKINS-1557 seems to be more concerned with coverage tools where the tests really are run twice. I guess we could debate whether those should be counted in the reports once or twice.

          However, there is a more obvious issue that a couple of comments touch upon here. If you run tests in both the test and integration-test phase, the ones in the test phase are counted twice. I believe that tpesce's hypothesis is correct. There is likely some trigger within Hudson that says, "Go read all test results" that is fired at the end of each of the two test phases. That would explain why the first test phase is read twice but the second once.

          By using that hypothesis, I came up with a fairly simple workaround. Just set the reportsDirectory to different places for each test phase. This worked for me. I used:

          <reportsDirectory>${project.build.directory}/surefire-reports/test</reportsDirectory>
          

          and

          <reportsDirectory>${project.build.directory}/surefire-reports/integration-test</reportsDirectory>
          

          John McNair added a comment - I think there are two separate issues here. JENKINS-1557 seems to be more concerned with coverage tools where the tests really are run twice. I guess we could debate whether those should be counted in the reports once or twice. However, there is a more obvious issue that a couple of comments touch upon here. If you run tests in both the test and integration-test phase, the ones in the test phase are counted twice. I believe that tpesce's hypothesis is correct. There is likely some trigger within Hudson that says, "Go read all test results" that is fired at the end of each of the two test phases. That would explain why the first test phase is read twice but the second once. By using that hypothesis, I came up with a fairly simple workaround. Just set the reportsDirectory to different places for each test phase. This worked for me. I used: <reportsDirectory>${project.build.directory}/surefire-reports/test</reportsDirectory> and <reportsDirectory>${project.build.directory}/surefire-reports/integration-test</reportsDirectory>

          John McNair added a comment -

          Here is an example with the workaround. In Hudson, this correctly reports two total tests. To expose the issue, comment out the two reportsDirectory lines. This will cause Hudson to report three total tests.

          John McNair added a comment - Here is an example with the workaround. In Hudson, this correctly reports two total tests. To expose the issue, comment out the two reportsDirectory lines. This will cause Hudson to report three total tests.
          John McNair made changes -
          Attachment New: hudson-junit.zip [ 19071 ]

            Unassigned Unassigned
            chrisgwarp Chris Graham
            Votes:
            28 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated: