I want to inform you that XML Reports of Surefire are not valid against XSD
      https://github.com/apache/maven-surefire/blob/master/maven-surefire-plugin/src/site/resources/xsd/surefire-test-report.xsd

      This should be fixed in Maven Surefire.
      If you use this XML and visualize the reports, pls adapt to the fix.
      https://issues.apache.org/jira/browse/SUREFIRE-1559

          [JENKINS-53237] Support Maven Surefire report version 3

          Tibor Digana created issue -
          Tibor Digana made changes -
          Issue Type Original: Bug [ 1 ] New: Improvement [ 4 ]

          Oleg Nenashev added a comment - - edited

          tibor17 Does SUREFIRE-1559 imply a breaking change in Maven Surefire?

          > This should be fixed in Maven Surefire. If you use this XML and visualize the reports, pls adapt to the fix.

          What is ETA of the fix? IIUC there is only an issue report without confirmed ETA or action plan (e.g. major version number bump, etc.). 

          Whatever the fix is needed on the Jenkins side, we will need to support both previous and new formats

          Oleg Nenashev added a comment - - edited tibor17 Does SUREFIRE-1559 imply a breaking change in Maven Surefire? > This should be fixed in Maven Surefire. If you use this XML and visualize the reports, pls adapt to the fix. What is ETA of the fix? IIUC there is only an issue report without confirmed ETA or action plan (e.g. major version number bump, etc.).  Whatever the fix is needed on the Jenkins side, we will need to support both previous and new formats

          Tibor Digana added a comment -

          That's the question because for me it breaks the form of report in something (only 4 elements I mentioned above) but the rest remains intact. If you do not mind in the stack trace system-out/err, then you would not see any difference.
          One way or another it was badly designed XSD/XML which could not be parsed and could not be valid. The fact that Jenkins plugins are parsing this XML without any problem means that they do not mind in the elements of (rerunError, rerunFailure, flakyFailure, flakyError). If they were used in Jenkins plugins and if the XML simple value was observed then it means that your users sometimes observed weird text in stack trace:
          <system-out>my console log...</system-out>

          Tibor Digana added a comment - That's the question because for me it breaks the form of report in something (only 4 elements I mentioned above) but the rest remains intact. If you do not mind in the stack trace system-out/err , then you would not see any difference. One way or another it was badly designed XSD/XML which could not be parsed and could not be valid. The fact that Jenkins plugins are parsing this XML without any problem means that they do not mind in the elements of ( rerunError, rerunFailure, flakyFailure, flakyError ). If they were used in Jenkins plugins and if the XML simple value was observed then it means that your users sometimes observed weird text in stack trace: <system-out>my console log...</system-out>

          Tibor Digana added a comment -

          These 4 elements are relatively new. Two were missing in XSD but present in XML.
          As I said bard work on our side.

          Tibor Digana added a comment - These 4 elements are relatively new. Two were missing in XSD but present in XML. As I said bard work on our side.

          Tibor Digana added a comment -

          oleg_nenashev
          It seems you agree since you have not reply. I am going to push it to master in ASF Maven. Any objections?

          Tibor Digana added a comment - oleg_nenashev It seems you agree since you have not reply. I am going to push it to master in ASF Maven. Any objections?
          Damian Szczepanik made changes -
          Assignee Original: Damian Szczepanik [ dszczepanik ] New: Tibor Digana [ tibor17 ]

          Nikolas Falco added a comment - - edited

          > tibor17The fact that Jenkins plugins are parsing this XML without any problem means that they do not mind in the elements of (rerunError, rerunFailure, flakyFailure, flakyError)

          Not true at all, plugins like xunit use the XSD to validate the XML (and with 2.22.1 is broken, files are skipped). It's not clear for me what is the gain of the stacktrace element and make rerun elements different from failure and error.

          EDIT: you missed to list the component that supports flaky elements... xunit and flaky-test-handler and for this reason you did get a response in time because no one of major interested people was informed. I'm here now because the new version had broken our organisation pipelines and I got this link from SUREFIRE-1559.

          Nikolas Falco added a comment - - edited > tibor17 The fact that Jenkins plugins are parsing this XML without any problem means that they do not mind in the elements of (rerunError, rerunFailure, flakyFailure, flakyError) Not true at all, plugins like xunit use the XSD to validate the XML (and with 2.22.1 is broken, files are skipped). It's not clear for me what is the gain of the stacktrace element and make rerun elements different from failure and error. EDIT: you missed to list the component that supports flaky elements... xunit and flaky-test-handler and for this reason you did get a response in time because no one of major interested people was informed. I'm here now because the new version had broken our organisation pipelines and I got this link from SUREFIRE-1559.
          Nikolas Falco made changes -
          Component/s New: flaky-test-handler-plugin [ 19823 ]
          Component/s New: xunit-plugin [ 15636 ]
          Nikolas Falco made changes -
          Assignee Original: Tibor Digana [ tibor17 ] New: Damian Szczepanik [ dszczepanik ]

            dszczepanik Damian Szczepanik
            tibor17 Tibor Digana
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: