• Type: Bug
• Resolution: Won't Do
• Priority: Major
• Component/s: xunit-plugin
• Labels:

  None
• Environment:

  Jenkins 2.277.2, xunit-plugin 3.0.1

[JENKINS-65438] xunit plugin fails to parse report from Catch2

Joachim Mairböck created issue - 2021-04-22 19:17

Joachim Mairböck made changes - 2021-04-22 19:19

Description

Original: The xUnit plugin fails to parse JUnit reports created by Catch2. The time (duration) values have the wrong format. The error message is:  {{{{WARNING: The file '<path>\TestResults-Scintilla.xml' is an invalid file. WARNING: At line 3 of file:/<path>/TestResults-Scintilla.xml:cvc-pattern-valid: Wert '0.022408' ist nicht Facet-gültig in Bezug auf Muster '(([0-9]\{0,3},)*[0-9]\{3}|[0-9]\{0,3})*(\.[0-9]\{0,3})?' für Typ 'SUREFIRE_TIME'. WARNING: At line 3 of file:/<path>/TestResults-Scintilla.xml:cvc-attribute.3: Wert '0.022408' des Attributs 'time' bei Element 'testsuite' hat keinen gültigen Typ 'SUREFIRE_TIME'. ERROR: Step ‘Publish xUnit test result report’ failed: The result file '<path>\TestResults-Scintilla.xml' for the metric 'JUnit' is not valid. The result file has been skipped.}}}} It seems that it expects time values only with 3 decimal digits, but the generated file contains 6. See the bug report for Catch2: https://github.com/catchorg/Catch2/issues/2221

New: The xUnit plugin fails to parse JUnit reports created by Catch2. The time (duration) values have the wrong format. The error message is: {noformat} WARNING: The file '<path>\TestResults-Scintilla.xml' is an invalid file. WARNING: At line 3 of [file:/|file:///]<path>/TestResults-Scintilla.xml:cvc-pattern-valid: Wert '0.022408' ist nicht Facet-gültig in Bezug auf Muster '(([0-9]{0,3},)*[0-9]{3}|[0-9]{0,3})*(\.[0-9]{0,3})?' für Typ 'SUREFIRE_TIME'. WARNING: At line 3 of [file:/|file:///]<path>/TestResults-Scintilla.xml:cvc-attribute.3: Wert '0.022408' des Attributs 'time' bei Element 'testsuite' hat keinen gültigen Typ 'SUREFIRE_TIME'. ERROR: Step ‘Publish xUnit test result report’ failed: The result file '<path>\TestResults-Scintilla.xml' for the metric 'JUnit' is not valid. The result file has been skipped. {noformat} It seems that it expects time values only with 3 decimal digits, but the generated file contains 6. See the bug report for Catch2: [https://github.com/catchorg/Catch2/issues/2221]

Collapse comment: Nikolas Falco added a comment - 2021-04-23 11:14, Edited by Nikolas Falco - 2021-04-23 11:15

Nikolas Falco added a comment - 2021-04-23 11:14 - edited

As reported in the plugin documentation JUnit supported schema are Ant junit and Maven Surefire

Schemas reports 3 digit for duration

Expand comment: Nikolas Falco added a comment - 2021-04-23 11:14, Edited by Nikolas Falco - 2021-04-23 11:15

Nikolas Falco added a comment - 2021-04-23 11:14 - edited As reported in the plugin  documentation JUnit supported schema are Ant junit and Maven Surefire Schemas reports 3 digit for duration

Nikolas Falco made changes - 2021-04-23 11:14

Resolution		New: Won't Do [ 10001 ]
Status	Original: Open [ 1 ]	New: Closed [ 6 ]

Collapse comment: Joachim Mairböck added a comment - 2021-04-23 14:28

Joachim Mairböck added a comment - 2021-04-23 14:28

I forgot to mention that this seems to be a regression of version 3.0. We used version 2.4.0 before on Jenkins 2.235.5 and it worked then.

We also used an older version of Catch before (version 1.0 build 47) that was part of the Scintilla version we are using. That version also misses the name attribute from the root testsuite element. I tried to fix it ad-hoc for the new xunit version by upgrading the Catch header, but the time format issue remained.

Expand comment: Joachim Mairböck added a comment - 2021-04-23 14:28

Joachim Mairböck added a comment - 2021-04-23 14:28 I forgot to mention that this seems to be a regression of version 3.0. We used version 2.4.0 before on Jenkins 2.235.5 and it worked then. We also used an older version of Catch before (version 1.0 build 47) that was part of the Scintilla version we are using. That version also misses the name attribute from the root testsuite element. I tried to fix it ad-hoc for the new xunit version by upgrading the Catch header, but the time format issue remained.

Collapse comment: Nikolas Falco added a comment - 2021-04-23 15:37

Nikolas Falco added a comment - 2021-04-23 15:37

Those schemas was used since version 2.0

Expand comment: Nikolas Falco added a comment - 2021-04-23 15:37

Nikolas Falco added a comment - 2021-04-23 15:37 Those schemas was used since version 2.0

Collapse comment: Joachim Mairböck added a comment - 2021-04-23 15:58

Joachim Mairböck added a comment - 2021-04-23 15:58

OK, sorry, I just checked. It was indeed broken before but I had not noticed that before. So it seems that this worked last before version 2.0 of xunit. I had to reconfigure the test result sources after the upgrade and forgot that I had the "ignore broken test results" flag checked before.

Expand comment: Joachim Mairböck added a comment - 2021-04-23 15:58

Joachim Mairböck added a comment - 2021-04-23 15:58 OK, sorry, I just checked. It was indeed broken before but I had not noticed that before. So it seems that this worked last before version 2.0 of xunit. I had to reconfigure the test result sources after the upgrade and forgot that I had the "ignore broken test results" flag checked before.

Mark Waite made changes - 2024-12-29 13:45

Comment

[ Spam comment ]

Mark Waite made changes - 2024-12-29 13:46

Comment

[ Spam comment ]