Not sure why you validate XML but anyway.

Since JUnit it's just a library and does not produce a report since 2.0 of xunit-plugin, supported JUnit report are those produced by the ant task and maven surefire. My XSD is compatible for both. I validate report in input to reduce possible unexpected parse/transformer (XSLT for not junit report) errors. This make easier resolve bugs because XSD filter out JUnit reports produced by other tools that declare to produce a correct report but they produces custom attributes and some times a different XML structure that could failure in the parser.

Parsing the XML and displaying the statistics is what has to be done by the plugins.

Yes but testsuite attributes meaningful depends from version.

What I wanted to say with At this point why do not add an attribute to mark the differences between flaky failures and failure also in the XML is that when rerun feature was introduced if had been added additional attribute for flaky failures (maybe also an attribute for runs) the XML would be full backward compatible at all because the attributes had the same meaningful of before. Now we have the root testsuite element that declare 10 tests and 2 failures when in real you have only 7 test method (but 10 runs...) and no failures.

<testsuite name="...FooTest" time="0" tests="7" errors="0" skipped="0" failures="0" runs="10" flakyfailures="2">

About the XSD version for what I can see for other software I consider a best practice publish XSD for each version and the XML declare the path to the specific XSD version

<testsuite ... xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report" ... /> (to be compatible with actual)
<testsuite ... xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-2.2.xsd" ... />
<testsuite ... xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd" ... />

If the change present in 3.0 version, we would have to wait quite long time and there would be a risk that the rerun feature as well as flaky failures are heavily used.

From the point of view of end user if I get an error updating from 2.x to 2.22.1 I have good reason to open a bug. In 3.x thanks to semantic versioning you can answer "we have changed the format". These kind of changes from 2.22.0 to 2.22.1 is much less reasonable, it looks more like a bug.

nfalco
If there are no objections I will introduce XML version in the XSD schema and we will proceed with plugin version 3.0.0-M1.
We will have multiple versions of this XML in web deployment and the old versions of plugin would be validated against the old schema.
I will open a pull request where you can have a look if you like and this way you will have the result under your control.
Is this fair?

For me it's ok

nfalco
We are near to Surefire 3.0.0-M2 release. I have made a fix for this issue. There is a new XSD version 3.0 and documentation explaining the decisions, see index.apt.vm. I have pushed a branch SUREFIRE-1590 on GitHub. Please review it. Thx.

It seems ok

nfalco
We are in release Vote for the version 3.0.0-M2
Please join us in the Vote:
https://lists.apache.org/thread.html/d44c7a8198e4f4d214211e40909249bcca67bdabd7d476731f2e1665@%3Cdev.maven.apache.org%3E

nfalco
The version 3.0.0-M2 is available in Maven Central.
Can you please adapt your plugin xUnit with the path of XSD?
It has a new location
https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd
and old location for legacy versions
https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd
I hope this would work for you. Sorry.

The XSD embedded in this plugin is not a copy of surefure XSD, it have to support old and new version and ant junit format version. So it is not an issue if the URl is changed but YES I have to check if latest version 3.0.0-M2 cause some issue to the actual plugin. Within this weekend I will perform the needed verification. Thank you.

tibor17 maybe it's the case also to open an issue to the flaky-test-handler-plugin that is the plugin that decorate JUnit reports with flaky tests in the XML.

If you want I could open for you.

nfalco
It's up to you. If you find an issue, I am all the ear.
But I do not know if your plugin can recognize the XSD version. I hope it can.