Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-14102

Unstable main build leads to post steps being executed even if configured not to

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      When a build "fails" due to test failures, and post steps are configured with the option "Run only if build succeeds", these post steps are still executed.

      (Jenkins 1.467)

        Attachments

          Activity

          Hide
          giulianobr Giuliano Ribeiro added a comment -

          I'm having the same issue on my builds.

          Please Jenkins' team take about it, it's so annoying .... dev team is complain a lot about it for me.

          Show
          giulianobr Giuliano Ribeiro added a comment - I'm having the same issue on my builds. Please Jenkins' team take about it, it's so annoying .... dev team is complain a lot about it for me.
          Hide
          ghudson Glenn Hudson added a comment -

          The issue is in;
          hudson.maven.reporters.SurefireArchiver
          (${jenkins}\maven-plugin\src\main\java\hudson\maven\reporters\SurefireArchiver.java)

          The awkward behavior since 2010 has been that if a unit test fails that the build is actually a SUCCESS. This has the result of the post steps always running if a unit test has failed.

          A potential way to fix this would be to add a configuration option (globally or for the build) that jobs with failed unit tests should not be "markAsSuccess".

          Code quality issue;
          MavenBuilder.markAsSuccess = true;
          Maven3Builder.markAsSuccess = true;
          An alternative to public static booleans should probably be found!

          Show
          ghudson Glenn Hudson added a comment - The issue is in; hudson.maven.reporters.SurefireArchiver (${jenkins}\maven-plugin\src\main\java\hudson\maven\reporters\SurefireArchiver.java) The awkward behavior since 2010 has been that if a unit test fails that the build is actually a SUCCESS. This has the result of the post steps always running if a unit test has failed. A potential way to fix this would be to add a configuration option (globally or for the build) that jobs with failed unit tests should not be "markAsSuccess". Code quality issue; MavenBuilder.markAsSuccess = true; Maven3Builder.markAsSuccess = true; An alternative to public static booleans should probably be found!
          Hide
          daha Daniel Haas added a comment -

          Hi everyone. I need this bug/feature so much that I'm willing to pay 50.00 bucks for it.
          This offer is registered at FreedomSponsors (http://www.freedomsponsors.org/core/issue/51/unstable-main-build-leads-to-post-steps-being-executed-even-if-configured-not-to).
          Once you solve it (according to the acceptance criteria described there), just create a FreedomSponsors account and mark it as resolved (oh, you'll need a Paypal account too)
          I'll then check it out and will gladly pay up!

          If anyone else would like to throw in a few bucks to elevate the priority on this issue, you should check out FreedomSponsors!

          Show
          daha Daniel Haas added a comment - Hi everyone. I need this bug/feature so much that I'm willing to pay 50.00 bucks for it. This offer is registered at FreedomSponsors ( http://www.freedomsponsors.org/core/issue/51/unstable-main-build-leads-to-post-steps-being-executed-even-if-configured-not-to ). Once you solve it (according to the acceptance criteria described there), just create a FreedomSponsors account and mark it as resolved (oh, you'll need a Paypal account too) I'll then check it out and will gladly pay up! If anyone else would like to throw in a few bucks to elevate the priority on this issue, you should check out FreedomSponsors!
          Hide
          advantiss Peter S added a comment - - edited

          please give more details - which post-build steps you use? or provide <publishers> section content of job's xml config file

          Show
          advantiss Peter S added a comment - - edited please give more details - which post-build steps you use? or provide <publishers> section content of job's xml config file
          Hide
          daha Daniel Haas added a comment -

          I'm executing a shell script afterwards. Actually what it is it should test first my build (maven build) and if the tests fail the script should be not executed.
          I put it into the Post Steps...

          Actually publishers is empty:

          <publishers/>

          There are stored postbuilders:

          <postbuilders>
          <hudson.tasks.Shell>
          <command>env && ./jdeploy $Server</command>
          </hudson.tasks.Shell>
          </postbuilders>

          In the jenkins web interface it is:

          Post Steps
          Run only if build succeeds
          Run only if build succeeds or is unstable
          Run regardless of build result

          Should the post-build steps run only for successful builds, etc.

          Execute shell

          Command env && ./jdeploy $Server

          Show
          daha Daniel Haas added a comment - I'm executing a shell script afterwards. Actually what it is it should test first my build (maven build) and if the tests fail the script should be not executed. I put it into the Post Steps... Actually publishers is empty: <publishers/> There are stored postbuilders: <postbuilders> <hudson.tasks.Shell> <command>env && ./jdeploy $Server</command> </hudson.tasks.Shell> </postbuilders> In the jenkins web interface it is: Post Steps Run only if build succeeds Run only if build succeeds or is unstable Run regardless of build result Should the post-build steps run only for successful builds, etc. Execute shell Command env && ./jdeploy $Server
          Hide
          tdemande Thomas Demande added a comment -

          I'm using "Invoke top-level Maven targets" post steps.

          Show
          tdemande Thomas Demande added a comment - I'm using "Invoke top-level Maven targets" post steps.
          Hide
          advantiss Peter S added a comment -

          Could you check, does <runPostStepsIfResult>success</runPostStepsIfResult> exists in job's xml config

          Show
          advantiss Peter S added a comment - Could you check, does <runPostStepsIfResult>success</runPostStepsIfResult> exists in job's xml config
          Hide
          tdemande Thomas Demande added a comment -

          I got this in the end of the job config file:

          <runPostStepsIfResult>
              <name>SUCCESS</name>
              <ordinal>0</ordinal>
              <color>BLUE</color>
            </runPostStepsIfResult>
          
          Show
          tdemande Thomas Demande added a comment - I got this in the end of the job config file: <runPostStepsIfResult> <name> SUCCESS </name> <ordinal> 0 </ordinal> <color> BLUE </color> </runPostStepsIfResult>
          Hide
          daha Daniel Haas added a comment -

          I have the same as Thomas

          Show
          daha Daniel Haas added a comment - I have the same as Thomas
          Hide
          advantiss Peter S added a comment -

          Could you place this system groovy script in postbuild steps as first step. Check console output after reproducing bug to see evaluated build result.

          system groovy script
          import jenkins.*;
          import jenkins.model.*;
          
          def build = Thread.currentThread().executable
          println('Build result:' + build.getResult().toString())
          
          Show
          advantiss Peter S added a comment - Could you place this system groovy script in postbuild steps as first step. Check console output after reproducing bug to see evaluated build result. system groovy script import jenkins.*; import jenkins.model.*; def build = Thread .currentThread().executable println( 'Build result:' + build.getResult().toString())
          Hide
          tdemande Thomas Demande added a comment - - edited

          Here's the output:

          C:\tools\jenkins\home\jobs\Test-validators\workspace>exit 0 
          Build result:UNSTABLE
          

          P.S.: Note that on that execution, post-steps were configured as "Run only if build succeeds"

          Show
          tdemande Thomas Demande added a comment - - edited Here's the output: C:\tools\jenkins\home\jobs\Test-validators\workspace>exit 0 Build result:UNSTABLE P.S.: Note that on that execution, post-steps were configured as "Run only if build succeeds"
          Hide
          maxime_lem Maxime Lemanissier added a comment -

          I get the same error with Jenkins 1.473, Maven build unstable but post steps still executed.
          I have also tried to pass a -Dmaven.test.failure.ignore=false, the build is then failed but post steps are still executed :/

          Is there any planned date/version to get a fix for this annoying problem?

          Show
          maxime_lem Maxime Lemanissier added a comment - I get the same error with Jenkins 1.473, Maven build unstable but post steps still executed. I have also tried to pass a -Dmaven.test.failure.ignore=false, the build is then failed but post steps are still executed :/ Is there any planned date/version to get a fix for this annoying problem?
          Hide
          domi Dominik Bartholdi added a comment -

          Even though the final build state of a maven project with test failures was correct or better the icon/ball had the correct color (yellow) - the build internally did not expose the correct state to post build steps.
          This causes issues with post build steps and publishers relying on the correct state of the build.

          just placed a pull request to fix this issue: https://github.com/jenkinsci/jenkins/pull/654

          Show
          domi Dominik Bartholdi added a comment - Even though the final build state of a maven project with test failures was correct or better the icon/ball had the correct color (yellow) - the build internally did not expose the correct state to post build steps. This causes issues with post build steps and publishers relying on the correct state of the build. just placed a pull request to fix this issue: https://github.com/jenkinsci/jenkins/pull/654
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: imod
          Path:
          maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java
          maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java
          maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          http://jenkins-ci.org/commit/jenkins/5f4f68be0081cd9b9554c7d0ce8529c573764056
          Log:
          [FIXED JENKINS-14102] mark maven build with test failures as unstable

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java http://jenkins-ci.org/commit/jenkins/5f4f68be0081cd9b9554c7d0ce8529c573764056 Log: [FIXED JENKINS-14102] mark maven build with test failures as unstable
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: imod
          Path:
          maven-plugin/src/main/java/hudson/maven/Maven2Builder.java
          maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          maven-plugin/src/main/java/hudson/maven/MavenBuild.java
          maven-plugin/src/main/java/hudson/maven/MavenBuilder.java
          maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          http://jenkins-ci.org/commit/jenkins/193173f5597ca02a5ce73a2e34343f77493025dc
          Log:
          [FIXED JENKINS-14102] fix also for maven2 and add test cases

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: maven-plugin/src/main/java/hudson/maven/Maven2Builder.java maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/MavenBuild.java maven-plugin/src/main/java/hudson/maven/MavenBuilder.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java http://jenkins-ci.org/commit/jenkins/193173f5597ca02a5ce73a2e34343f77493025dc Log: [FIXED JENKINS-14102] fix also for maven2 and add test cases
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #2156
          [FIXED JENKINS-14102] mark maven build with test failures as unstable (Revision 5f4f68be0081cd9b9554c7d0ce8529c573764056)
          [FIXED JENKINS-14102] fix also for maven2 and add test cases (Revision 193173f5597ca02a5ce73a2e34343f77493025dc)

          Result = SUCCESS
          imod : 5f4f68be0081cd9b9554c7d0ce8529c573764056
          Files :

          • maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          • maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          • test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          • maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java
          • maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java

          imod : 193173f5597ca02a5ce73a2e34343f77493025dc
          Files :

          • test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java
          • maven-plugin/src/main/java/hudson/maven/Maven3Builder.java
          • maven-plugin/src/main/java/hudson/maven/MavenBuild.java
          • maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java
          • maven-plugin/src/main/java/hudson/maven/MavenBuilder.java
          • maven-plugin/src/main/java/hudson/maven/Maven2Builder.java
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2156 [FIXED JENKINS-14102] mark maven build with test failures as unstable (Revision 5f4f68be0081cd9b9554c7d0ce8529c573764056) [FIXED JENKINS-14102] fix also for maven2 and add test cases (Revision 193173f5597ca02a5ce73a2e34343f77493025dc) Result = SUCCESS imod : 5f4f68be0081cd9b9554c7d0ce8529c573764056 Files : maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java maven-plugin/src/main/java/hudson/maven/reporters/SurefireArchiver.java maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java imod : 193173f5597ca02a5ce73a2e34343f77493025dc Files : test/src/test/java/hudson/maven/MavenBuildSurefireFailedTest.java maven-plugin/src/main/java/hudson/maven/Maven3Builder.java maven-plugin/src/main/java/hudson/maven/MavenBuild.java maven-plugin/src/main/java/hudson/maven/reporters/TestFailureDetector.java maven-plugin/src/main/java/hudson/maven/MavenBuilder.java maven-plugin/src/main/java/hudson/maven/Maven2Builder.java
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: imod
          Path:
          src/main/java/hudson/maven/Maven3Builder.java
          src/main/java/hudson/maven/MavenModuleSet.java
          src/main/java/hudson/maven/reporters/SurefireArchiver.java
          src/main/java/hudson/maven/reporters/TestFailureDetector.java
          http://jenkins-ci.org/commit/maven-plugin/adf9fc658df6e18a0d0648fea011bbe752094558
          Log:
          [FIXED JENKINS-14102] mark maven build with test failures as unstable

          Originally-Committed-As: 5f4f68be0081cd9b9554c7d0ce8529c573764056

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: src/main/java/hudson/maven/Maven3Builder.java src/main/java/hudson/maven/MavenModuleSet.java src/main/java/hudson/maven/reporters/SurefireArchiver.java src/main/java/hudson/maven/reporters/TestFailureDetector.java http://jenkins-ci.org/commit/maven-plugin/adf9fc658df6e18a0d0648fea011bbe752094558 Log: [FIXED JENKINS-14102] mark maven build with test failures as unstable Originally-Committed-As: 5f4f68be0081cd9b9554c7d0ce8529c573764056
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: imod
          Path:
          src/main/java/hudson/maven/Maven2Builder.java
          src/main/java/hudson/maven/Maven3Builder.java
          src/main/java/hudson/maven/MavenBuild.java
          src/main/java/hudson/maven/MavenBuilder.java
          src/main/java/hudson/maven/reporters/TestFailureDetector.java
          http://jenkins-ci.org/commit/maven-plugin/bb10503a662676b8e72991f63d68859004ea9587
          Log:
          [FIXED JENKINS-14102] fix also for maven2 and add test cases

          Originally-Committed-As: 193173f5597ca02a5ce73a2e34343f77493025dc

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: imod Path: src/main/java/hudson/maven/Maven2Builder.java src/main/java/hudson/maven/Maven3Builder.java src/main/java/hudson/maven/MavenBuild.java src/main/java/hudson/maven/MavenBuilder.java src/main/java/hudson/maven/reporters/TestFailureDetector.java http://jenkins-ci.org/commit/maven-plugin/bb10503a662676b8e72991f63d68859004ea9587 Log: [FIXED JENKINS-14102] fix also for maven2 and add test cases Originally-Committed-As: 193173f5597ca02a5ce73a2e34343f77493025dc

            People

            Assignee:
            abayer Andrew Bayer
            Reporter:
            tdemande Thomas Demande
            Votes:
            7 Vote for this issue
            Watchers:
            11 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: