Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29565

xUnit plugin does not fail job when parsing multiple results files

XMLWordPrintable

      When the xUnit plugin Custom Tool option is parsing data files it does not correctly identify failing tests when processing thresholds.

      For example:

      Test Set 1 - no failures
      Test Set 2 - some failures
      Test Set 3 - no failures

      Results in identified failures for Test Set 2 but the job still succeeds after processing thresholds. Please see the attached console log output and screenshot of the jenkins job.

      All threshold values in the plugin were not filled (believe this to mean no failed tests are permitted).

        1. JenkinsJobScreenshot.jpg
          JenkinsJobScreenshot.jpg
          74 kB
        2. xUnitBug.txt
          0.6 kB

          [JENKINS-29565] xUnit plugin does not fail job when parsing multiple results files

          Nikolas Falco added a comment -

          All threshold values in the plugin were not filled (believe this to mean no failed tests are permitted).

          Wrong assumption. The description cite:
          "Configure the build status. A build is considered as unstable or failure if the new or total number of failed tests exceeds the specified thresholds.", empty does not means 0. If empty means 0 when are all empty what is the default bahavior? unstable or failure?

          The image toolstip describes the meaning of each single threshold. Every toolstip end with: "Leave this field empty if the unstable state of the build should not depend on the total number of failed tests"

          Nikolas Falco added a comment - All threshold values in the plugin were not filled (believe this to mean no failed tests are permitted). Wrong assumption. The description cite: "Configure the build status. A build is considered as unstable or failure if the new or total number of failed tests exceeds the specified thresholds.", empty does not means 0. If empty means 0 when are all empty what is the default bahavior? unstable or failure? The image toolstip describes the meaning of each single threshold. Every toolstip end with: "Leave this field empty if the unstable state of the build should not depend on the total number of failed tests"

            nfalco Nikolas Falco
            drtaddei Dave Taddei
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: