Details
-
Improvement
-
Status: Resolved (View Workflow)
-
Minor
-
Resolution: Fixed
-
None
Description
Currently if 20% or more of tests fail, the weather for a project will be downgraded (cloudy).
Some users have a large number of tests and one test failure is a major issue.
The request is to allow customization of the % of Junit test failures that will cause a weather downgrade. Users that specify 100% pass require all tests to pass to maintain sunny weather.
Attachments
Activity
Mark Sinclair
added a comment - It appears I'm not getting an accurate health report out of Junit.
I am running Junit version 1.2 (the health amplification factor fix is in).
The problem I am seeing is I don't see the weather report changing, it is always reporting 100% from Junit. I tried different settings the amplification factor, but get the same result, always 100% health. I created a job with only 1 test in it, Junit reports the test as failing, and it shows up in my trend graph. Everything else with Junit seems to be working.
The weather is reported sunny with 1 test failing out of a total of 1 test 100% health. The build is reported as unstable, but the weather is sunny. I expect the health/weather to be 0% from Junit because 100% of the tests are failing.
Mark Sinclair
added a comment - It appears I'm not getting an accurate health report out of Junit.
I am running Junit version 1.2 (the health amplification factor fix is in).
The problem I am seeing is I don't see the weather report changing, it is always reporting 100% from Junit. I tried different settings the amplification factor, but get the same result, always 100% health. I created a job with only 1 test in it, Junit reports the test as failing, and it shows up in my trend graph. Everything else with Junit seems to be working.
The weather is reported sunny with 1 test failing out of a total of 1 test 100% health. The build is reported as unstable, but the weather is sunny. I expect the health/weather to be 0% from Junit because 100% of the tests are failing.
msinclair I would suggest you file a separate bug and mark it blocking this one using a JIRA link. I know nothing about the intended behavior of healthScaleFactory. I was merely fixing the problem in its configuration form.
Mark Sinclair
added a comment - Thanks Jesse,
I also put up a question on google groups to see if I'm doing something wrong in my xml generation:
https://groups.google.com/d/topic/jenkinsci-users/-StbbZ0s8jI/discussion
Mark Sinclair
added a comment - Thanks Jesse,
I also put up a question on google groups to see if I'm doing something wrong in my xml generation:
https://groups.google.com/d/topic/jenkinsci-users/-StbbZ0s8jI/discussion Mark Sinclair
added a comment - The always 100% health report issue is tracked by the following:
https://issues.jenkins-ci.org/browse/JENKINS-25573
Mark Sinclair
added a comment -
The always 100% health report issue is tracked by the following:
https://issues.jenkins-ci.org/browse/JENKINS-25573
Code changed in jenkins
User: Jesse Glick
Path:
src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
src/main/resources/hudson/tasks/junit/JUnitResultArchiver/config.jelly
src/test/java/hudson/tasks/junit/JUnitResultArchiverTest.java
src/test/resources/hudson/tasks/junit/JUnitResultArchiverTest/MockTestDataPublisher/config.jelly
http://jenkins-ci.org/commit/junit-plugin/e6416e34c0987a839d3dc2a34938dd1d2ac0d035
Log:
[FIXED JENKINS-24006] Changes to healthScaleFactor were not actually being saved.
Ripping out the archaic manual form binding code and replacing with the modern simplified scheme.