Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64471

warnings-ng tool for nunit

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We have tried to record results of nunit2 execution via warnings-ng plugin. Unfortunately we haven't found a ready to use warnings-ng tool for nunit (we are still using nunit2 and not the nunit3).
      So we have used Microsoft tool (mssql.exe) with the xsl-convertion taken from https://github.com/jenkinsci/nunit-plugin/tree/master/src/main/resources/hudson/plugins/nunit (and added the encoding "utf-8" parameter to generate junit-files, which can be consumed by xunit-plugin).
      The resulting files could be successfully integrated into the jenkins build reports using xunit-plugin:

      xunit ( tools: [JUnit(pattern: '.out\\*.Test.junit-result.xml', stopProcessingIfError: true)] )

      But we cannot see any issues in the warnings-ng plugin (using reportIssues as follows):
      recordIssues(
                          name: 'Unit Tests',
                          enabledForFailure: true,
                          aggregatingResults: false,
                          tools: [junitParser(id: 'unitTests', name: 'UnitTests', pattern: '.out\\*.Test.junit-result.xml', reportEncoding: 'UTF-8')]
                        )

        Attachments

          Activity

          Hide
          smorita Shin-ichi Morita added a comment -

          It seems to have a couple of issues:

          • NUnit is not supported by warnings-ng-plugin.
          • Reusing nunit-plugin's xsl seems not working.

          Regarding the former, please consult with wargings-ng-plugin.

          According to https://github.com/jenkinsci/warnings-ng-plugin/blob/master/doc/Documentation.md#creating-support-for-a-custom-tool, you can create a custom parser, although I'm not familiar with it.

          Regarding the latter, could you tell whether your nunit2 report is successfully processed by nunit-plugin or not?

           

          Regards,

          Show
          smorita Shin-ichi Morita added a comment - It seems to have a couple of issues: NUnit is not supported by warnings-ng-plugin. Reusing nunit-plugin's xsl seems not working. Regarding the former, please consult with wargings-ng-plugin. According to https://github.com/jenkinsci/warnings-ng-plugin/blob/master/doc/Documentation.md#creating-support-for-a-custom-tool,  you can create a custom parser, although I'm not familiar with it. Regarding the latter, could you tell whether your nunit2 report is successfully processed by nunit-plugin or not?   Regards,
          Hide
          drulli Ulli Hafner added a comment -

          The warnings plugin is not intended to show unit test results, so I removed the component from this issue.

          Show
          drulli Ulli Hafner added a comment - The warnings plugin is not intended to show unit test results, so I removed the component from this issue.
          Hide
          mrm927 Rashid added a comment -

          @Ulli Hafner, what is then the intent to provide junitParser in the warnings-ng plugin, if it doesn't consider failed and skipped tests?

          Our expectation was, that we could get some statistics about the failed / skipped tests via warnings-ng plugin.

          The nunit-plugin can of cause import the nunit-reports.
          Even xunit-plugin is able to import the manually converted "junit-reports", made of nunit-report as described in the description of the issue.
          Our issue was importing the results into warnings-ng

          Show
          mrm927 Rashid added a comment - @ Ulli Hafner , what is then the intent to provide junitParser in the warnings-ng plugin, if it doesn't consider failed and skipped tests? Our expectation was, that we could get some statistics about the failed / skipped tests via warnings-ng plugin. The nunit-plugin can of cause import the nunit-reports. Even xunit-plugin is able to import the manually converted "junit-reports", made of nunit-report as described in the description of the issue. Our issue was importing the results into warnings-ng
          Hide
          drulli Ulli Hafner added a comment -

          The intend was to get some features of the warnings plugin for free that were not available in the JUnit plugin (basically the JS chart and delta computation).
          (And another internal reason: I use the API to obtain the number of passed, skipped and failed tests in my autograding module.)

          What are you missing in the JUnit plugin?

          Show
          drulli Ulli Hafner added a comment - The intend was to get some features of the warnings plugin for free that were not available in the JUnit plugin (basically the JS chart and delta computation). (And another internal reason: I use the API to obtain the number of passed, skipped and failed tests in my autograding module.) What are you missing in the JUnit plugin?
          Hide
          tkachenko Konstantin Tkachenko added a comment -

          I've attached a sample build report from our jenkins. The build has 2 failed tests (imported from junit-reports via xunit plugin), but the warnings-ng plugin hasn't found any warnings using junitParser (for the same reports).

          That was exactly the reason why we tried to use warnings-ng plugin for our nunit-tests - we were trying to get chart and some statistics about skipped and failed tests.

          Show
          tkachenko Konstantin Tkachenko added a comment - I've attached a sample build report from our jenkins. The build has 2 failed tests (imported from junit-reports via xunit plugin), but the warnings-ng plugin hasn't found any warnings using junitParser (for the same reports). That was exactly the reason why we tried to use warnings-ng plugin for our nunit-tests - we were trying to get chart and some statistics about skipped and failed tests.
          Hide
          drulli Ulli Hafner added a comment -

          If your file is not in JUnit format it is clear that it does not work with the warnings plugin, I am using an external library to parse those files. Those things that are done in the xunit are not part of the warnings plugin parser.

          Show
          drulli Ulli Hafner added a comment - If your file is not in JUnit format it is clear that it does not work with the warnings plugin, I am using an external library to parse those files. Those things that are done in the xunit are not part of the warnings plugin parser.

            People

            Assignee:
            smorita Shin-ichi Morita
            Reporter:
            mrm927 Rashid
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated: