-
Bug
-
Resolution: Fixed
-
Major
-
Powered by SuggestiMate
When a build "fails" due to test failures, and post steps are configured with the option "Run only if build succeeds", these post steps are still executed.
(Jenkins 1.467)
[JENKINS-14102] Unstable main build leads to post steps being executed even if configured not to
The issue is in;
hudson.maven.reporters.SurefireArchiver
(${jenkins}\maven-plugin\src\main\java\hudson\maven\reporters\SurefireArchiver.java)
The awkward behavior since 2010 has been that if a unit test fails that the build is actually a SUCCESS. This has the result of the post steps always running if a unit test has failed.
A potential way to fix this would be to add a configuration option (globally or for the build) that jobs with failed unit tests should not be "markAsSuccess".
Code quality issue;
MavenBuilder.markAsSuccess = true;
Maven3Builder.markAsSuccess = true;
An alternative to public static booleans should probably be found!
Hi everyone. I need this bug/feature so much that I'm willing to pay 50.00 bucks for it.
This offer is registered at FreedomSponsors (http://www.freedomsponsors.org/core/issue/51/unstable-main-build-leads-to-post-steps-being-executed-even-if-configured-not-to).
Once you solve it (according to the acceptance criteria described there), just create a FreedomSponsors account and mark it as resolved (oh, you'll need a Paypal account too)
I'll then check it out and will gladly pay up!
If anyone else would like to throw in a few bucks to elevate the priority on this issue, you should check out FreedomSponsors!
please give more details - which post-build steps you use? or provide <publishers> section content of job's xml config file
I'm executing a shell script afterwards. Actually what it is it should test first my build (maven build) and if the tests fail the script should be not executed.
I put it into the Post Steps...
Actually publishers is empty:
<publishers/>
There are stored postbuilders:
<postbuilders>
<hudson.tasks.Shell>
<command>env && ./jdeploy $Server</command>
</hudson.tasks.Shell>
</postbuilders>
In the jenkins web interface it is:
Post Steps
Run only if build succeeds
Run only if build succeeds or is unstable
Run regardless of build result
Should the post-build steps run only for successful builds, etc.
Execute shell
Command env && ./jdeploy $Server
Could you check, does <runPostStepsIfResult>success</runPostStepsIfResult> exists in job's xml config
I got this in the end of the job config file:
<runPostStepsIfResult> <name>SUCCESS</name> <ordinal>0</ordinal> <color>BLUE</color> </runPostStepsIfResult>
Could you place this system groovy script in postbuild steps as first step. Check console output after reproducing bug to see evaluated build result.
import jenkins.*; import jenkins.model.*; def build = Thread.currentThread().executable println('Build result:' + build.getResult().toString())
Here's the output:
C:\tools\jenkins\home\jobs\Test-validators\workspace>exit 0 Build result:UNSTABLE
P.S.: Note that on that execution, post-steps were configured as "Run only if build succeeds"
I get the same error with Jenkins 1.473, Maven build unstable but post steps still executed.
I have also tried to pass a -Dmaven.test.failure.ignore=false, the build is then failed but post steps are still executed :/
Is there any planned date/version to get a fix for this annoying problem?
Even though the final build state of a maven project with test failures was correct or better the icon/ball had the correct color (yellow) - the build internally did not expose the correct state to post build steps.
This causes issues with post build steps and publishers relying on the correct state of the build.
just placed a pull request to fix this issue: https://github.com/jenkinsci/jenkins/pull/654
Code changed in jenkins
User: imod
Path:
maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java
maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java
maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
http://jenkins-ci.org/commit/jenkins/5f4f68be0081cd9b9554c7d0ce8529c573764056
Log:
[FIXED JENKINS-14102] mark maven build with test failures as unstable
Code changed in jenkins
User: imod
Path:
maven-plugin/src/main/java/hudson/maven/Maven2Builder.java
maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
maven-plugin/src/main/java/hudson/maven/MavenBuild.java
maven-plugin/src/main/java/hudson/maven/MavenBuilder.java
maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
http://jenkins-ci.org/commit/jenkins/193173f5597ca02a5ce73a2e34343f77493025dc
Log:
[FIXED JENKINS-14102] fix also for maven2 and add test cases
Integrated in jenkins_main_trunk #2156
[FIXED JENKINS-14102] mark maven build with test failures as unstable (Revision 5f4f68be0081cd9b9554c7d0ce8529c573764056)
[FIXED JENKINS-14102] fix also for maven2 and add test cases (Revision 193173f5597ca02a5ce73a2e34343f77493025dc)
Result = SUCCESS
imod : 5f4f68be0081cd9b9554c7d0ce8529c573764056
Files :
- maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
- maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
- test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
- maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java
- maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java
imod : 193173f5597ca02a5ce73a2e34343f77493025dc
Files :
- test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
- maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
- maven-plugin/src/main/java/hudson/maven/MavenBuild.java
- maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
- maven-plugin/src/main/java/hudson/maven/MavenBuilder.java
- maven-plugin/src/main/java/hudson/maven/Maven2Builder.java
Code changed in jenkins
User: imod
Path:
src/main/java/hudson/maven/Maven3Builder.java
src/main/java/hudson/maven/MavenModuleSet.java
src/main/java/hudson/maven/reporters/SurefireArchiver.java
src/main/java/hudson/maven/reporters/TestFailureDetector.java
http://jenkins-ci.org/commit/maven-plugin/adf9fc658df6e18a0d0648fea011bbe752094558
Log:
[FIXED JENKINS-14102] mark maven build with test failures as unstable
Originally-Committed-As: 5f4f68be0081cd9b9554c7d0ce8529c573764056
Code changed in jenkins
User: imod
Path:
src/main/java/hudson/maven/Maven2Builder.java
src/main/java/hudson/maven/Maven3Builder.java
src/main/java/hudson/maven/MavenBuild.java
src/main/java/hudson/maven/MavenBuilder.java
src/main/java/hudson/maven/reporters/TestFailureDetector.java
http://jenkins-ci.org/commit/maven-plugin/bb10503a662676b8e72991f63d68859004ea9587
Log:
[FIXED JENKINS-14102] fix also for maven2 and add test cases
Originally-Committed-As: 193173f5597ca02a5ce73a2e34343f77493025dc
I'm having the same issue on my builds.
Please Jenkins' team take about it, it's so annoying .... dev team is complain a lot about it for me.