-
Bug
-
Resolution: Not A Defect
-
Major
-
Jenkins ver. 2.150.1
xUnit plugin 2.3.2
Ubuntu 16.04.5 LTS
Project Using Nightwatch 1.0.18
Using declarative pipeline jenkinsfile I have the following:
xunit testDataPublishers: [[$class: 'AutomateTestDataPublisher']],
thresholds: [failed(failureNewThreshold: '0', failureThreshold: '0', unstableNewThreshold: '0', unstableThreshold: '0')],
tools: [Custom(customXSL: 'test/e2e/custom-to-junit.xsl', deleteOutputFiles: true, failIfNotNew: true, pattern: 'test/e2e/reports/**/*.xml', skipNoTestFiles: false, stopProcessingIfError: true)]
The custom-to-junit.xsl file contains the recommended XSL from here [^custom-to-junit.xsl] as linked to form the plugin wiki page.
The project under test is using Nightwatch for e2e front end testing. The XML report file generated by Night watch contains the following snippet:
<testcase name="default e2e tests" classname="test" time="13.87" assertions="4">}}{{<failure message="Timed out while waiting for element <.user-dropdown> to be present for 10000 milliseconds. - expected "visible" but got: "not found""> at Object.defaultE2eTests [as default e2e tests] (/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/test/e2e/specs/test.js:19:8)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)</failure>
<system-out>[[ATTACHMENT|/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/test/e2e/reports/screenshots/test/default-e2e-tests_FAILED_Jan-21-2019-145635-GMT-0800.png]]</system-out>
<failure message=" at Object.defaultE2eTests [as default e2e tests] (/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/test/e2e/specs/test.js:19:8)"> at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)</failure>
</testcase>
Once xUnit transforms the file the resulting XML file contains the following transformed snippet:
<testcase name="default e2e tests" time="13.870" classname="test">}}{{<failure message="Timed out while waiting for element <.user-dropdown> to be present for 10000 milliseconds. - expected "visible" but got: "not found""><![CDATA[ at Object.defaultE2eTests [as default e2e tests] (/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/test/e2e/specs/test.js:19:8)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)]]></failure>
<system-out><![CDATA[[[ATTACHMENT|/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/test/e2e/reports/screenshots/test/default-e2e-tests_FAILED_Jan-21-2019-145635-GMT-0800.png]]]]></system-out>
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)
</testcase>
This results in the following error in the build log file:
INFO: Processing Custom Tool
INFO: [Custom Tool] - 2 test report file(s) were found with the pattern 'test/e2e/reports/*/.xml' relative to '/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ' for the testing framework 'Custom Tool'.
WARNING: The converted file for the input file '/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/test/e2e/reports/CHROME_71.0.3578.80_XP_test.xml' doesnt match the JUnit format.
WARNING: At line 18 of file:/var/lib/jenkins/workspace/infra-sms-ui_browserstack-ZV4ZESG7HBLPJAFOXOTRI7VWSYQOL2FCXEE7I66MFQZVXY4TAFDQ/generatedJUnitFiles/8d717da3-a72e-4782-abb7-2a76fbdcd9b2/Custom%20Tool/TEST-f653ef00-1ca8-402f-a18e-1008b363542e.xml:cvc-complex-type.2.3: Element 'testcase' cannot have character [children], because the type's content type is element-only.
ERROR: cvc-complex-type.2.3: Element 'testcase' cannot have character [children], because the type's content type is element-only.
This error appears to be caused by the transform not properly removing the second instance of a <failure></failure> element. It removes the element tags but not the contents of the element.
[JENKINS-55716] xUnit Custom XSL not Transforming Files properly
nfalco Yes, that is the XSL file I intended to link to. The issue is not with the XSL itself but with how it is applied to the report file. It appears that the code that executes the transformation is not properly remove some elements.
For example the following snippet from a test report file (some details removed for simplicity)
<testcase name="" classname="" time="" assertions="">
<failure message="">
stacktrace here
</failure>
<system-out>
Screenshot attachment
</system-out>
<failure message=" ">
Stacktrace #2 here
</failure>
</testcase>
is transformed into
<testcase name="" classname="" time="" assertions="">
<failure message="">
stacktrace here
</failure>
<system-out>
Screenshot attachment
</system-out>
Stacktrace #2 here
</testcase>
This results in the blod stack trace being part of the testcase which results in the the error "Element 'testcase' cannot have character [children], because the type's content type is element-only".
It is my impression that the second stack trace should be removed with the second failure element becvause it is surrounded by the <failure></failure> tags.
The link of custom XSL is broken but I suppose you would link to this one. That XSL is given as starting point for customisation. The defect is not related to the plugin.
If you post the source report that cause the issue I will have a look.