Details
-
Type:
Bug
-
Status: Open (View Workflow)
-
Priority:
Minor
-
Resolution: Unresolved
-
Component/s: nunit-plugin, pipeline
-
Labels:None
-
Similar Issues:
Description
We have tried to record results of nunit2 execution via warnings-ng plugin. Unfortunately we haven't found a ready to use warnings-ng tool for nunit (we are still using nunit2 and not the nunit3).
So we have used Microsoft tool (mssql.exe) with the xsl-convertion taken from https://github.com/jenkinsci/nunit-plugin/tree/master/src/main/resources/hudson/plugins/nunit (and added the encoding "utf-8" parameter to generate junit-files, which can be consumed by xunit-plugin).
The resulting files could be successfully integrated into the jenkins build reports using xunit-plugin:
xunit ( tools: [JUnit(pattern: '.out\\*.Test.junit-result.xml', stopProcessingIfError: true)] )
But we cannot see any issues in the warnings-ng plugin (using reportIssues as follows):
recordIssues(
name: 'Unit Tests',
enabledForFailure: true,
aggregatingResults: false,
tools: [junitParser(id: 'unitTests', name: 'UnitTests', pattern: '.out\\*.Test.junit-result.xml', reportEncoding: 'UTF-8')]
)
Attachments
Activity
The warnings plugin is not intended to show unit test results, so I removed the component from this issue.
@Ulli Hafner, what is then the intent to provide junitParser in the warnings-ng plugin, if it doesn't consider failed and skipped tests?
Our expectation was, that we could get some statistics about the failed / skipped tests via warnings-ng plugin.
The nunit-plugin can of cause import the nunit-reports.
Even xunit-plugin is able to import the manually converted "junit-reports", made of nunit-report as described in the description of the issue.
Our issue was importing the results into warnings-ng
Rashid
added a comment - @ Ulli Hafner , what is then the intent to provide junitParser in the warnings-ng plugin, if it doesn't consider failed and skipped tests?
Our expectation was, that we could get some statistics about the failed / skipped tests via warnings-ng plugin.
The nunit-plugin can of cause import the nunit-reports.
Even xunit-plugin is able to import the manually converted "junit-reports", made of nunit-report as described in the description of the issue.
Our issue was importing the results into warnings-ng The intend was to get some features of the warnings plugin for free that were not available in the JUnit plugin (basically the JS chart and delta computation).
(And another internal reason: I use the API to obtain the number of passed, skipped and failed tests in my autograding module.)
What are you missing in the JUnit plugin?
I've attached a sample build report from our jenkins. The build has 2 failed tests (imported from junit-reports via xunit plugin), but the warnings-ng plugin hasn't found any warnings using junitParser (for the same reports).
That was exactly the reason why we tried to use warnings-ng plugin for our nunit-tests - we were trying to get chart and some statistics about skipped and failed tests.
If your file is not in JUnit format it is clear that it does not work with the warnings plugin, I am using an external library to parse those files. Those things that are done in the xunit are not part of the warnings plugin parser.
It seems to have a couple of issues:
Regarding the former, please consult with wargings-ng-plugin.
According to https://github.com/jenkinsci/warnings-ng-plugin/blob/master/doc/Documentation.md#creating-support-for-a-custom-tool, you can create a custom parser, although I'm not familiar with it.
Regarding the latter, could you tell whether your nunit2 report is successfully processed by nunit-plugin or not?
Regards,