Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29251

Exclude skipped tests from the failed count

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • test-stability-plugin
    • None

      It seems that a test result that is skipped is included in the failure count. For example I have a test that has 15 failures and 3 skips, the stability status say: "Failed 17 times in the last 22 runs.". That does not seem correct to me since a skip is not really the same as a failure.

      I think it should either be excluded from the count or at least make it configurable how the skips should be counted.

          [JENKINS-29251] Exclude skipped tests from the failed count

          kutzi added a comment -

          15 + 3 would be 18 failures, so that doesn't add up.

          The logic of the plugin should be to leave the test history completely unchanged if the test was skipped (which makes sense IMO). So neither the failed cound nor the total count is changed by it.

          Does that match your observation: i.e. do you have 17 failures in the last 22 build where this test actually did run?

          kutzi added a comment - 15 + 3 would be 18 failures, so that doesn't add up. The logic of the plugin should be to leave the test history completely unchanged if the test was skipped (which makes sense IMO). So neither the failed cound nor the total count is changed by it. Does that match your observation: i.e. do you have 17 failures in the last 22 build where this test actually did run?

            kutzi kutzi
            zitrax Daniel Bengtsson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: