Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6545

JUnit report parsing cannot handle nested suites

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • junit-plugin
    • None

      JUnit allows to nest testsuites indefinetly (using TestSuite.addTestSuite()) which results in xml files that have <testsuites> element as root and under that a hierarchy of <testsuite> elements. Currently the SuiteResult class in Hudson can only handle a single level of nesting, which results in the "None of the test reports contained any result" error in the build.

      So my junit result looks like this:

      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites name="Automated Tests" tests="186" errors="0" failures="0" ignored="0">
        <testsuite name="test.AutomatedTests" time="4.486">
          <testsuite name="test.fs.FileSystemTest" time="0.014">
            <testcase name="testPrefix" classname="test.fs.FileSystemTest" time="0.003"/>
          </testsuite>
          <testsuite name="test.db.DatabaseTest" time="0.014">
            <testcase name="testTable" classname="test.db.DatabaseTest" time="0.003"/>
          </testsuite>
        </testsuite>
      </testsuites
      

          [JENKINS-6545] JUnit report parsing cannot handle nested suites

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545

          I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit.
          Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks?

          For example ant <junit> xml formatter flattens test suites, doesn't keep nesting.
          Maven surefire does the same.
          Even tried TestNG wrapped around JUnit with the same result.
          (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17)

          So which is the test runner that would generate such a nested report for a nested JUnit suite?

          Andrei Costescu added a comment - I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit. Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks? For example ant <junit> xml formatter flattens test suites, doesn't keep nesting. Maven surefire does the same. Even tried TestNG wrapped around JUnit with the same result. (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17) So which is the test runner that would generate such a nested report for a nested JUnit suite?

          Andrei Costescu added a comment - - edited

          Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml).
          So I need one that can be used in Jenkins.

          Andrei Costescu added a comment - - edited Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml). So I need one that can be used in Jenkins.

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins-test-harness/c9f1e5f27a87a26ff7e6b86e0adb1d6f77dfc948
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Originally-Committed-As: 97f1d02cbdc0aa85d2e907d878b126e80fd0880b

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins-test-harness/c9f1e5f27a87a26ff7e6b86e0adb1d6f77dfc948 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Originally-Committed-As: 97f1d02cbdc0aa85d2e907d878b126e80fd0880b

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins-test-harness/fd2c85b6aeb64003677d5faa6c2cdfc32c4f6a14
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Originally-Committed-As: d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins-test-harness/fd2c85b6aeb64003677d5faa6c2cdfc32c4f6a14 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Originally-Committed-As: d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9

          Code changed in jenkins
          User: Lionel Orry
          Path:
          src/test/java/hudson/tasks/junit/SuiteResultTest.java
          http://jenkins-ci.org/commit/junit-plugin/d4c888e06ada7dbe5841c0b5894c812f6bda6d23
          Log:
          Fix a referenced issue number

          The issue number for nested test suites is JENKINS-6545, not JENKINS-6454.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Lionel Orry Path: src/test/java/hudson/tasks/junit/SuiteResultTest.java http://jenkins-ci.org/commit/junit-plugin/d4c888e06ada7dbe5841c0b5894c812f6bda6d23 Log: Fix a referenced issue number The issue number for nested test suites is JENKINS-6545 , not JENKINS-6454 .

            Unassigned Unassigned
            nukleon nukleon
            Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: