-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
Jenkins 2.2
TestNG Results Plugin 1.12
After updating the TestNG plugin to 1.12, the new feature from JENKINS-20985 caused our tests to pass regardless of failures/skips in our TestNG results. By default, the threshold is set to 100% if users have not configured the plugin yet.
* Helps resolve XML configs for versions before 1.11 when these new config options were introduced. * See https://wiki.jenkins-ci.org/display/JENKINS/Hint+on+retaining+backward+compatibility * @return resolved object */ protected Object readResolve() { if (unstableSkips == null) { unstableSkips = unstableOnSkippedTests ? 0 : 100; } if (failedFails == null) { failedFails = 100; } if (failedSkips == null) { failedSkips = 100; } if (thresholdMode == null) { thresholdMode = 2; } return this; }
This means that by default, a test will require MORE THAN 100% skips/failures in order to mark the build as unstable/failed from TestNG results. It would make more sense if the default was set to 0, and the thresholdMode set to 1. This way, a single failure/skip will cause the build to be unstable/fail like in previous versions.
- is related to
-
JENKINS-34711 NullPointerException after Update to 1.11 of plugin
- Closed
-
JENKINS-20985 TestNG plugin should has a possibility to use thresholds for failed tests
- Closed
- relates to
-
JENKINS-42819 Percentage Threshold Is Broken for 100%
- Open