Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-14102

Unstable main build leads to post steps being executed even if configured not to

      When a build "fails" due to test failures, and post steps are configured with the option "Run only if build succeeds", these post steps are still executed.

      (Jenkins 1.467)

          [JENKINS-14102] Unstable main build leads to post steps being executed even if configured not to

          I'm having the same issue on my builds.

          Please Jenkins' team take about it, it's so annoying .... dev team is complain a lot about it for me.

          Giuliano Ribeiro added a comment - I'm having the same issue on my builds. Please Jenkins' team take about it, it's so annoying .... dev team is complain a lot about it for me.

          Glenn Hudson added a comment -

          The issue is in;
          hudson.maven.reporters.SurefireArchiver
          (${jenkins}\maven-plugin\src\main\java\hudson\maven\reporters\SurefireArchiver.java)

          The awkward behavior since 2010 has been that if a unit test fails that the build is actually a SUCCESS. This has the result of the post steps always running if a unit test has failed.

          A potential way to fix this would be to add a configuration option (globally or for the build) that jobs with failed unit tests should not be "markAsSuccess".

          Code quality issue;
          MavenBuilder.markAsSuccess = true;
          Maven3Builder.markAsSuccess = true;
          An alternative to public static booleans should probably be found!

          Glenn Hudson added a comment - The issue is in; hudson.maven.reporters.SurefireArchiver (${jenkins}\maven-plugin\src\main\java\hudson\maven\reporters\SurefireArchiver.java) The awkward behavior since 2010 has been that if a unit test fails that the build is actually a SUCCESS. This has the result of the post steps always running if a unit test has failed. A potential way to fix this would be to add a configuration option (globally or for the build) that jobs with failed unit tests should not be "markAsSuccess". Code quality issue; MavenBuilder.markAsSuccess = true; Maven3Builder.markAsSuccess = true; An alternative to public static booleans should probably be found!

          Daniel Haas added a comment -

          Hi everyone. I need this bug/feature so much that I'm willing to pay 50.00 bucks for it.
          This offer is registered at FreedomSponsors (http://www.freedomsponsors.org/core/issue/51/unstable-main-build-leads-to-post-steps-being-executed-even-if-configured-not-to).
          Once you solve it (according to the acceptance criteria described there), just create a FreedomSponsors account and mark it as resolved (oh, you'll need a Paypal account too)
          I'll then check it out and will gladly pay up!

          If anyone else would like to throw in a few bucks to elevate the priority on this issue, you should check out FreedomSponsors!

          Daniel Haas added a comment - Hi everyone. I need this bug/feature so much that I'm willing to pay 50.00 bucks for it. This offer is registered at FreedomSponsors ( http://www.freedomsponsors.org/core/issue/51/unstable-main-build-leads-to-post-steps-being-executed-even-if-configured-not-to ). Once you solve it (according to the acceptance criteria described there), just create a FreedomSponsors account and mark it as resolved (oh, you'll need a Paypal account too) I'll then check it out and will gladly pay up! If anyone else would like to throw in a few bucks to elevate the priority on this issue, you should check out FreedomSponsors!

          Peter S added a comment - - edited

          please give more details - which post-build steps you use? or provide <publishers> section content of job's xml config file

          Peter S added a comment - - edited please give more details - which post-build steps you use? or provide <publishers> section content of job's xml config file

          Daniel Haas added a comment -

          I'm executing a shell script afterwards. Actually what it is it should test first my build (maven build) and if the tests fail the script should be not executed.
          I put it into the Post Steps...

          Actually publishers is empty:

          <publishers/>

          There are stored postbuilders:

          <postbuilders>
          <hudson.tasks.Shell>
          <command>env && ./jdeploy $Server</command>
          </hudson.tasks.Shell>
          </postbuilders>

          In the jenkins web interface it is:

          Post Steps
          Run only if build succeeds
          Run only if build succeeds or is unstable
          Run regardless of build result

          Should the post-build steps run only for successful builds, etc.

          Execute shell

          Command env && ./jdeploy $Server

          Daniel Haas added a comment - I'm executing a shell script afterwards. Actually what it is it should test first my build (maven build) and if the tests fail the script should be not executed. I put it into the Post Steps... Actually publishers is empty: <publishers/> There are stored postbuilders: <postbuilders> <hudson.tasks.Shell> <command>env && ./jdeploy $Server</command> </hudson.tasks.Shell> </postbuilders> In the jenkins web interface it is: Post Steps Run only if build succeeds Run only if build succeeds or is unstable Run regardless of build result Should the post-build steps run only for successful builds, etc. Execute shell Command env && ./jdeploy $Server

          I'm using "Invoke top-level Maven targets" post steps.

          Thomas Demande added a comment - I'm using "Invoke top-level Maven targets" post steps.

          Peter S added a comment -

          Could you check, does <runPostStepsIfResult>success</runPostStepsIfResult> exists in job's xml config

          Peter S added a comment - Could you check, does <runPostStepsIfResult>success</runPostStepsIfResult> exists in job's xml config

          I got this in the end of the job config file:

          <runPostStepsIfResult>
              <name>SUCCESS</name>
              <ordinal>0</ordinal>
              <color>BLUE</color>
            </runPostStepsIfResult>
          

          Thomas Demande added a comment - I got this in the end of the job config file: <runPostStepsIfResult> <name> SUCCESS </name> <ordinal> 0 </ordinal> <color> BLUE </color> </runPostStepsIfResult>

          Daniel Haas added a comment -

          I have the same as Thomas

          Daniel Haas added a comment - I have the same as Thomas

          Peter S added a comment -

          Could you place this system groovy script in postbuild steps as first step. Check console output after reproducing bug to see evaluated build result.

          system groovy script
          import jenkins.*;
          import jenkins.model.*;
          
          def build = Thread.currentThread().executable
          println('Build result:' + build.getResult().toString())
          

          Peter S added a comment - Could you place this system groovy script in postbuild steps as first step. Check console output after reproducing bug to see evaluated build result. system groovy script import jenkins.*; import jenkins.model.*; def build = Thread .currentThread().executable println( 'Build result:' + build.getResult().toString())

          Thomas Demande added a comment - - edited

          Here's the output:

          C:\tools\jenkins\home\jobs\Test-validators\workspace>exit 0 
          Build result:UNSTABLE
          

          P.S.: Note that on that execution, post-steps were configured as "Run only if build succeeds"

          Thomas Demande added a comment - - edited Here's the output: C:\tools\jenkins\home\jobs\Test-validators\workspace>exit 0 Build result:UNSTABLE P.S.: Note that on that execution, post-steps were configured as "Run only if build succeeds"

          I get the same error with Jenkins 1.473, Maven build unstable but post steps still executed.
          I have also tried to pass a -Dmaven.test.failure.ignore=false, the build is then failed but post steps are still executed :/

          Is there any planned date/version to get a fix for this annoying problem?

          Maxime Lemanissier added a comment - I get the same error with Jenkins 1.473, Maven build unstable but post steps still executed. I have also tried to pass a -Dmaven.test.failure.ignore=false, the build is then failed but post steps are still executed :/ Is there any planned date/version to get a fix for this annoying problem?

          Even though the final build state of a maven project with test failures was correct or better the icon/ball had the correct color (yellow) - the build internally did not expose the correct state to post build steps.
          This causes issues with post build steps and publishers relying on the correct state of the build.

          just placed a pull request to fix this issue: https://github.com/jenkinsci/jenkins/pull/654

          Dominik Bartholdi added a comment - Even though the final build state of a maven project with test failures was correct or better the icon/ball had the correct color (yellow) - the build internally did not expose the correct state to post build steps. This causes issues with post build steps and publishers relying on the correct state of the build. just placed a pull request to fix this issue: https://github.com/jenkinsci/jenkins/pull/654

          Code changed in jenkins
          User: imod
          Path:
          maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java
          maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java
          maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          http://jenkins-ci.org/commit/jenkins/5f4f68be0081cd9b9554c7d0ce8529c573764056
          Log:
          [FIXED JENKINS-14102] mark maven build with test failures as unstable

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java http://jenkins-ci.org/commit/jenkins/5f4f68be0081cd9b9554c7d0ce8529c573764056 Log: [FIXED JENKINS-14102] mark maven build with test failures as unstable

          Code changed in jenkins
          User: imod
          Path:
          maven-plugin/src/main/java/hudson/maven/Maven2Builder.java
          maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          maven-plugin/src/main/java/hudson/maven/MavenBuild.java
          maven-plugin/src/main/java/hudson/maven/MavenBuilder.java
          maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          http://jenkins-ci.org/commit/jenkins/193173f5597ca02a5ce73a2e34343f77493025dc
          Log:
          [FIXED JENKINS-14102] fix also for maven2 and add test cases

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: maven-plugin/src/main/java/hudson/maven/Maven2Builder.java maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/MavenBuild.java maven-plugin/src/main/java/hudson/maven/MavenBuilder.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java http://jenkins-ci.org/commit/jenkins/193173f5597ca02a5ce73a2e34343f77493025dc Log: [FIXED JENKINS-14102] fix also for maven2 and add test cases

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2156
          [FIXED JENKINS-14102] mark maven build with test failures as unstable (Revision 5f4f68be0081cd9b9554c7d0ce8529c573764056)
          [FIXED JENKINS-14102] fix also for maven2 and add test cases (Revision 193173f5597ca02a5ce73a2e34343f77493025dc)

          Result = SUCCESS
          imod : 5f4f68be0081cd9b9554c7d0ce8529c573764056
          Files :

          • maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          • maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          • test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          • maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java
          • maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java

          imod : 193173f5597ca02a5ce73a2e34343f77493025dc
          Files :

          • test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          • maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          • maven-plugin/src/main/java/hudson/maven/MavenBuild.java
          • maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          • maven-plugin/src/main/java/hudson/maven/MavenBuilder.java
          • maven-plugin/src/main/java/hudson/maven/Maven2Builder.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2156 [FIXED JENKINS-14102] mark maven build with test failures as unstable (Revision 5f4f68be0081cd9b9554c7d0ce8529c573764056) [FIXED JENKINS-14102] fix also for maven2 and add test cases (Revision 193173f5597ca02a5ce73a2e34343f77493025dc) Result = SUCCESS imod : 5f4f68be0081cd9b9554c7d0ce8529c573764056 Files : maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java imod : 193173f5597ca02a5ce73a2e34343f77493025dc Files : test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/MavenBuild.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java maven-plugin/src/main/java/hudson/maven/MavenBuilder.java maven-plugin/src/main/java/hudson/maven/Maven2Builder.java

          Code changed in jenkins
          User: imod
          Path:
          src/main/java/hudson/maven/Maven3Builder.java
          src/main/java/hudson/maven/MavenModuleSet.java
          src/main/java/hudson/maven/reporters/SurefireArchiver.java
          src/main/java/hudson/maven/reporters/TestFailureDetector.java
          http://jenkins-ci.org/commit/maven-plugin/adf9fc658df6e18a0d0648fea011bbe752094558
          Log:
          [FIXED JENKINS-14102] mark maven build with test failures as unstable

          Originally-Committed-As: 5f4f68be0081cd9b9554c7d0ce8529c573764056

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: src/main/java/hudson/maven/Maven3Builder.java src/main/java/hudson/maven/MavenModuleSet.java src/main/java/hudson/maven/reporters/SurefireArchiver.java src/main/java/hudson/maven/reporters/TestFailureDetector.java http://jenkins-ci.org/commit/maven-plugin/adf9fc658df6e18a0d0648fea011bbe752094558 Log: [FIXED JENKINS-14102] mark maven build with test failures as unstable Originally-Committed-As: 5f4f68be0081cd9b9554c7d0ce8529c573764056

          Code changed in jenkins
          User: imod
          Path:
          src/main/java/hudson/maven/Maven2Builder.java
          src/main/java/hudson/maven/Maven3Builder.java
          src/main/java/hudson/maven/MavenBuild.java
          src/main/java/hudson/maven/MavenBuilder.java
          src/main/java/hudson/maven/reporters/TestFailureDetector.java
          http://jenkins-ci.org/commit/maven-plugin/bb10503a662676b8e72991f63d68859004ea9587
          Log:
          [FIXED JENKINS-14102] fix also for maven2 and add test cases

          Originally-Committed-As: 193173f5597ca02a5ce73a2e34343f77493025dc

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: src/main/java/hudson/maven/Maven2Builder.java src/main/java/hudson/maven/Maven3Builder.java src/main/java/hudson/maven/MavenBuild.java src/main/java/hudson/maven/MavenBuilder.java src/main/java/hudson/maven/reporters/TestFailureDetector.java http://jenkins-ci.org/commit/maven-plugin/bb10503a662676b8e72991f63d68859004ea9587 Log: [FIXED JENKINS-14102] fix also for maven2 and add test cases Originally-Committed-As: 193173f5597ca02a5ce73a2e34343f77493025dc

            abayer Andrew Bayer
            tdemande Thomas Demande
            Votes:
            7 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: