Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6545

JUnit report parsing cannot handle nested suites

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • junit-plugin
    • None

      JUnit allows to nest testsuites indefinetly (using TestSuite.addTestSuite()) which results in xml files that have <testsuites> element as root and under that a hierarchy of <testsuite> elements. Currently the SuiteResult class in Hudson can only handle a single level of nesting, which results in the "None of the test reports contained any result" error in the build.

      So my junit result looks like this:

      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites name="Automated Tests" tests="186" errors="0" failures="0" ignored="0">
        <testsuite name="test.AutomatedTests" time="4.486">
          <testsuite name="test.fs.FileSystemTest" time="0.014">
            <testcase name="testPrefix" classname="test.fs.FileSystemTest" time="0.003"/>
          </testsuite>
          <testsuite name="test.db.DatabaseTest" time="0.014">
            <testcase name="testTable" classname="test.db.DatabaseTest" time="0.003"/>
          </testsuite>
        </testsuite>
      </testsuites
      

          [JENKINS-6545] JUnit report parsing cannot handle nested suites

          ewysp added a comment -

          Try the xUnit plug in. I ran into the same issue as you, and found this plugin to suit our needs ... for now.

          http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin

          ewysp added a comment - Try the xUnit plug in. I ran into the same issue as you, and found this plugin to suit our needs ... for now. http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin

          MichaelBreu added a comment -

          Hello ewvsg,

          It would be helpful if you could include here your XSLT-file needed for the xUnit-Workaround.

          Michael

          MichaelBreu added a comment - Hello ewvsg, It would be helpful if you could include here your XSLT-file needed for the xUnit-Workaround. Michael

          Here is a description of the XSL needed to transform the nested junit xml output into something which can be parsed by Jenkins.

          http://stackoverflow.com/questions/4640300/hudson-ci-and-phpunit-none-of-the-test-reports-contained-any-result

          Russell Francis added a comment - Here is a description of the XSL needed to transform the nested junit xml output into something which can be parsed by Jenkins. http://stackoverflow.com/questions/4640300/hudson-ci-and-phpunit-none-of-the-test-reports-contained-any-result

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be
          Log:
          [FIXED JENKINS-6545] support nested test suites

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be Log: [FIXED JENKINS-6545] support nested test suites

          dogfood added a comment -

          Integrated in jenkins_main_trunk #648
          [FIXED JENKINS-6545] support nested test suites

          Kohsuke Kawaguchi : 4129ac50a39ae2522c20ce73e2b10872a07c04be
          Files :

          • core/src/main/java/hudson/tasks/junit/SuiteResult.java
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #648 [FIXED JENKINS-6545] support nested test suites Kohsuke Kawaguchi : 4129ac50a39ae2522c20ce73e2b10872a07c04be Files : core/src/main/java/hudson/tasks/junit/SuiteResult.java changelog.html

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          dogfood added a comment -

          Integrated in jenkins_main_trunk #649
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Stephen Connolly : 97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Files :

          • test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java

          dogfood added a comment - Integrated in jenkins_main_trunk #649 Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Stephen Connolly : 97f1d02cbdc0aa85d2e907d878b126e80fd0880b Files : test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java

          This is my fix, at least you could mention my name in the changeset... since you copied it from the Hudson changeset i made few days ago

          https://github.com/hudson/hudson/commit/3d79ac89df3a3107904fa3b91b5d86075c86a5dd

          I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...

          Victor Garcia Perez added a comment - This is my fix, at least you could mention my name in the changeset... since you copied it from the Hudson changeset i made few days ago https://github.com/hudson/hudson/commit/3d79ac89df3a3107904fa3b91b5d86075c86a5dd I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...

          I'm sorry you felt that your code was taken without credit.

          You might not believe this, but I really did not copy your change. The pull request in Hudson caught my attention, so I looked at the bug, and the fix seemed simple enough, so I went ahead. I think you'd agree that it's hard to fix this bug in any other way.

          With that said, this is a sensitive problem, and I don't want to be accused of stealing the changes, so I've added your authorship in the copyright header to show my respect: https://github.com/jenkinsci/jenkins/commit/46994700e4d2b3a5ea84607035bb954ed3c4c750

          Kohsuke Kawaguchi added a comment - I'm sorry you felt that your code was taken without credit. You might not believe this, but I really did not copy your change. The pull request in Hudson caught my attention, so I looked at the bug, and the fix seemed simple enough, so I went ahead. I think you'd agree that it's hard to fix this bug in any other way. With that said, this is a sensitive problem, and I don't want to be accused of stealing the changes, so I've added your authorship in the copyright header to show my respect: https://github.com/jenkinsci/jenkins/commit/46994700e4d2b3a5ea84607035bb954ed3c4c750

          I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list...
          

          Forgot to reply to the last topic. I'm sorry that we failed to respond to your inquiry — I just checked the dev archive but couldn't find your post — did it actually get delivered? Can you tell me the subject?

          Currently, sending in a pull request is the best way to get an attention on patches. I hope this didn't discourage you from continuing to contribute to Jenkins.

          Kohsuke Kawaguchi added a comment - I wanted to commit the fix to Jenkins but nobody answer me about how was the procedure in the developers list... Forgot to reply to the last topic. I'm sorry that we failed to respond to your inquiry — I just checked the dev archive but couldn't find your post — did it actually get delivered? Can you tell me the subject? Currently, sending in a pull request is the best way to get an attention on patches. I hope this didn't discourage you from continuing to contribute to Jenkins.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #734
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Kohsuke Kawaguchi : d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Files :

          • test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java

          dogfood added a comment - Integrated in jenkins_main_trunk #734 Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Kohsuke Kawaguchi : d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Files : test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          Compare: https://github.com/jenkinsci/jenkins/compare/3192dfb...073bd72

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545 Compare: https://github.com/jenkinsci/jenkins/compare/3192dfb...073bd72

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be
          Log:
          [FIXED JENKINS-6545] support nested test suites

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4129ac50a39ae2522c20ce73e2b10872a07c04be Log: [FIXED JENKINS-6545] support nested test suites

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/97f1d02cbdc0aa85d2e907d878b126e80fd0880b Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins/d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/test/java/hudson/tasks/junit/SuiteResultTest.java
          core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml
          http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2
          Log:
          finally added a test for JENKINS-6545

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/test/java/hudson/tasks/junit/SuiteResultTest.java core/src/test/resources/hudson/tasks/junit/junit-report-nested-testsuites.xml http://jenkins-ci.org/commit/jenkins/073bd72fc904a965eb63880f1e87d0626721b0b2 Log: finally added a test for JENKINS-6545

          I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit.
          Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks?

          For example ant <junit> xml formatter flattens test suites, doesn't keep nesting.
          Maven surefire does the same.
          Even tried TestNG wrapped around JUnit with the same result.
          (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17)

          So which is the test runner that would generate such a nested report for a nested JUnit suite?

          Andrei Costescu added a comment - I've been searching for a while for a tool that would generate this kind of nested <testsuite> report for JUnit. Did any of you actually have such a nested report xml generated from nested JUnit tests suites (either 3.x or 4.x)? Or it it only generated by other testing frameworks? For example ant <junit> xml formatter flattens test suites, doesn't keep nesting. Maven surefire does the same. Even tried TestNG wrapped around JUnit with the same result. (junit 4.11 (used 3 like style and 4 like style), testNG 6.8, ant 1.9.4, maven-surefire-plugin 2.17) So which is the test runner that would generate such a nested report for a nested JUnit suite?

          Andrei Costescu added a comment - - edited

          Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml).
          So I need one that can be used in Jenkins.

          Andrei Costescu added a comment - - edited Actually I found one but not very useful for Jenkins (the eclipse jUnit results view has an Export in a menu that will generate such xml). So I need one that can be used in Jenkins.

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins-test-harness/c9f1e5f27a87a26ff7e6b86e0adb1d6f77dfc948
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Originally-Committed-As: 97f1d02cbdc0aa85d2e907d878b126e80fd0880b

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins-test-harness/c9f1e5f27a87a26ff7e6b86e0adb1d6f77dfc948 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Originally-Committed-As: 97f1d02cbdc0aa85d2e907d878b126e80fd0880b

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java
          http://jenkins-ci.org/commit/jenkins-test-harness/fd2c85b6aeb64003677d5faa6c2cdfc32c4f6a14
          Log:
          Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545

          Originally-Committed-As: d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: test/src/main/java/org/jvnet/hudson/test/TestExtensionLoader.java http://jenkins-ci.org/commit/jenkins-test-harness/fd2c85b6aeb64003677d5faa6c2cdfc32c4f6a14 Log: Tsk Tsk! Kohsuke you missed a possible NPE for people running hpi:run on plugins in fixing JENKINS-6545 Originally-Committed-As: d68e3ba2c51b730c04d9c15d82ac012e8d1aaec9

          Code changed in jenkins
          User: Lionel Orry
          Path:
          src/test/java/hudson/tasks/junit/SuiteResultTest.java
          http://jenkins-ci.org/commit/junit-plugin/d4c888e06ada7dbe5841c0b5894c812f6bda6d23
          Log:
          Fix a referenced issue number

          The issue number for nested test suites is JENKINS-6545, not JENKINS-6454.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Lionel Orry Path: src/test/java/hudson/tasks/junit/SuiteResultTest.java http://jenkins-ci.org/commit/junit-plugin/d4c888e06ada7dbe5841c0b5894c812f6bda6d23 Log: Fix a referenced issue number The issue number for nested test suites is JENKINS-6545 , not JENKINS-6454 .

            Unassigned Unassigned
            nukleon nukleon
            Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: