Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24006

Customize number of Junit test failures that affect weather

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • junit-plugin
    • None

      Currently if 20% or more of tests fail, the weather for a project will be downgraded (cloudy).

      Some users have a large number of tests and one test failure is a major issue.

      The request is to allow customization of the % of Junit test failures that will cause a weather downgrade. Users that specify 100% pass require all tests to pass to maintain sunny weather.

          [JENKINS-24006] Customize number of Junit test failures that affect weather

          Daniel Beck added a comment -

          Daniel Beck added a comment - Fixed in https://github.com/jenkinsci/jenkins/commit/17fdfab3cd43b03732f322162fac16df87c604f1 by Stephen Connolly. Scheduled for 1.576.

          Mark Sinclair added a comment -

          Thanks for implementing this.

          I tried it out and have an issue. After saving a job with a new amplification factor, going back to the config screen always loads the default 1.0 amplification factor, rather than the saved value.

          It may be due to the code below:

          core/src/main/java/hudson/tasks/test/AbstractTestResultAction.java:163

          public double getHealthScaleFactor() {
          return 1.0;

          Mark Sinclair added a comment - Thanks for implementing this. I tried it out and have an issue. After saving a job with a new amplification factor, going back to the config screen always loads the default 1.0 amplification factor, rather than the saved value. It may be due to the code below: core/src/main/java/hudson/tasks/test/AbstractTestResultAction.java:163 public double getHealthScaleFactor() { return 1.0;

          Daniel Beck added a comment -

          Why did you assign this to me? I even wrote that Stephen implemented this...

          Daniel Beck added a comment - Why did you assign this to me? I even wrote that Stephen implemented this...

          Mark Sinclair added a comment -

          My mistake - will reassign

          Mark Sinclair added a comment - My mistake - will reassign

          Scott Busche added a comment -

          I've created a PR that fixes the issue Mark is seeing - https://github.com/jenkinsci/junit-plugin/pull/4

          Scott Busche added a comment - I've created a PR that fixes the issue Mark is seeing - https://github.com/jenkinsci/junit-plugin/pull/4

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
          src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly
          src/test/java/hudson/tasks/junit/JUnitResultArchiverTest.java
          src/test/resources/hudson/tasks/junit/JUnitResultArchiverTest/MockTestDataPublisher/config.jelly
          http://jenkins-ci.org/commit/junit-plugin/e6416e34c0987a839d3dc2a34938dd1d2ac0d035
          Log:
          [FIXED JENKINS-24006] Changes to healthScaleFactor were not actually being saved.
          Ripping out the archaic manual form binding code and replacing with the modern simplified scheme.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/tasks/junit/JUnitResultArchiver.java src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly src/test/java/hudson/tasks/junit/JUnitResultArchiverTest.java src/test/resources/hudson/tasks/junit/JUnitResultArchiverTest/MockTestDataPublisher/config.jelly http://jenkins-ci.org/commit/junit-plugin/e6416e34c0987a839d3dc2a34938dd1d2ac0d035 Log: [FIXED JENKINS-24006] Changes to healthScaleFactor were not actually being saved. Ripping out the archaic manual form binding code and replacing with the modern simplified scheme.

          Mark Sinclair added a comment -

          It appears I'm not getting an accurate health report out of Junit.

          I am running Junit version 1.2 (the health amplification factor fix is in).

          The problem I am seeing is I don't see the weather report changing, it is always reporting 100% from Junit. I tried different settings the amplification factor, but get the same result, always 100% health. I created a job with only 1 test in it, Junit reports the test as failing, and it shows up in my trend graph. Everything else with Junit seems to be working.

          The weather is reported sunny with 1 test failing out of a total of 1 test 100% health. The build is reported as unstable, but the weather is sunny. I expect the health/weather to be 0% from Junit because 100% of the tests are failing.

          Mark Sinclair added a comment - It appears I'm not getting an accurate health report out of Junit. I am running Junit version 1.2 (the health amplification factor fix is in). The problem I am seeing is I don't see the weather report changing, it is always reporting 100% from Junit. I tried different settings the amplification factor, but get the same result, always 100% health. I created a job with only 1 test in it, Junit reports the test as failing, and it shows up in my trend graph. Everything else with Junit seems to be working. The weather is reported sunny with 1 test failing out of a total of 1 test 100% health. The build is reported as unstable, but the weather is sunny. I expect the health/weather to be 0% from Junit because 100% of the tests are failing.

          Jesse Glick added a comment -

          msinclair I would suggest you file a separate bug and mark it blocking this one using a JIRA link. I know nothing about the intended behavior of healthScaleFactory. I was merely fixing the problem in its configuration form.

          Jesse Glick added a comment - msinclair I would suggest you file a separate bug and mark it blocking this one using a JIRA link. I know nothing about the intended behavior of healthScaleFactory . I was merely fixing the problem in its configuration form.

          Mark Sinclair added a comment -

          Thanks Jesse,

          I also put up a question on google groups to see if I'm doing something wrong in my xml generation:
          https://groups.google.com/d/topic/jenkinsci-users/-StbbZ0s8jI/discussion

          Mark Sinclair added a comment - Thanks Jesse, I also put up a question on google groups to see if I'm doing something wrong in my xml generation: https://groups.google.com/d/topic/jenkinsci-users/-StbbZ0s8jI/discussion

          Mark Sinclair added a comment -

          The always 100% health report issue is tracked by the following:
          https://issues.jenkins-ci.org/browse/JENKINS-25573

          Mark Sinclair added a comment - The always 100% health report issue is tracked by the following: https://issues.jenkins-ci.org/browse/JENKINS-25573

            jglick Jesse Glick
            msinclair Mark Sinclair
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: