Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24832

Failed maven builds using -T are showing up as Aborted

    XMLWordPrintable

Details

    Description

      After upgrading to Jenkins 1.565.2, I'm seeing some builds showing up with status Aborted instead of Failed when there is a compilation issue. We use maven jobs and have incremental building enabled (this has worked fine for us for a long time.) We also use multiple threads for the maven build. Seems if one thread is in the middle of running unit tests where as the other one compiles and fails, then the other thread gets killed and status for module shows as ABORTED in main server log. Could this be caused by https://issues.jenkins-ci.org/browse/JENKINS-22641 fix introduced in latest LTS release?

      [woot@eng301 logs]$ grep 13179 catalina.out | grep apps-nonstop
      INFO: SCM changes detected in apps-nonstop. Triggering  #13179
      INFO: apps-nonstop/com.netbase:deploy #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-prospero-widget #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-stream-server #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:prospero-framework #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:transformer #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-cb-closure #13179 main build action completed: ABORTED
      INFO: apps-nonstop/com.netbase:dailyupdate #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:persist-api #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:alert-manager #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:content-api #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-iodocs #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:persist-cb #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:log #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:rosalind #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:acc-block-result #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:content-rosalind #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-ariel #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:consumerbase2 #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-ix #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:nlp #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:test #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:cb-insight-api-tools #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:app-cb #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:prospero-api #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:prospero-lucene4 #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:applications #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:megatron #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop/com.netbase:engine-lucene4 #13179 main build action completed: NOT_BUILT
      INFO: apps-nonstop #13179 main build action completed: FAILURE
      

      Attachments

        Issue Links

          Activity

            swolk swolk created issue -
            swolk swolk added a comment -

            Changing to critical as this makes it so folks don't get notified on build failures. I can add a build notification for Aborted but it is confusing to users.

            swolk swolk added a comment - Changing to critical as this makes it so folks don't get notified on build failures. I can add a build notification for Aborted but it is confusing to users.
            swolk swolk made changes -
            Field Original Value New Value
            Priority Major [ 3 ] Critical [ 2 ]
            danielbeck Daniel Beck added a comment -

            Which version were you upgrading from?

            danielbeck Daniel Beck added a comment - Which version were you upgrading from?
            swolk swolk added a comment -

            We were previously running 1.565.1

            swolk swolk added a comment - We were previously running 1.565.1
            danielbeck Daniel Beck added a comment -

            Is this something that could be reproduced by compiling e.g. some Maven-based open source project?

            danielbeck Daniel Beck added a comment - Is this something that could be reproduced by compiling e.g. some Maven-based open source project?
            danielbeck Daniel Beck made changes -
            Component/s maven [ 16033 ]
            swolk swolk added a comment -

            I can try if you can point me at a maven multi-module project

            swolk swolk added a comment - I can try if you can point me at a maven multi-module project
            swolk swolk added a comment - - edited

            OK, this is very easy to reproduce. Create a maven job for this example multi-module project https://github.com/idodevjobs/sample-multi-module-project. For root pom, use parent module. Add -T4 for Maven Goals and Options to enable multiple threads. Kick off the build and 'sample-multi-module-web Maven Webapp' module should fail because of missing dependency. Job status will show as Aborted instead of Failed.

            swolk swolk added a comment - - edited OK, this is very easy to reproduce. Create a maven job for this example multi-module project https://github.com/idodevjobs/sample-multi-module-project . For root pom, use parent module. Add -T4 for Maven Goals and Options to enable multiple threads. Kick off the build and 'sample-multi-module-web Maven Webapp' module should fail because of missing dependency. Job status will show as Aborted instead of Failed.
            danielbeck Daniel Beck added a comment - - edited

            (Instructions failed to mention I actually need a goal...)

            I get one failed, one successful, and one aborted module, and the overall build status is aborted. Jenkins 1.580(ish), Maven plugin 2.6, building on the CentOS 6 Jenkins master node. So at least it's reproducible.

            danielbeck Daniel Beck added a comment - - edited (Instructions failed to mention I actually need a goal...) I get one failed, one successful, and one aborted module, and the overall build status is aborted. Jenkins 1.580(ish), Maven plugin 2.6, building on the CentOS 6 Jenkins master node. So at least it's reproducible.
            swolk swolk made changes -
            Summary Failed builds are showing up as Aborted Failed maven builds using -T are showing up as Aborted
            swolk swolk made changes -
            Attachment screenshot-1.jpg [ 26941 ]
            swolk swolk added a comment -

            Sorry about that. Yep, that is my result as well.

            swolk swolk added a comment - Sorry about that. Yep, that is my result as well.
            swolk swolk added a comment - - edited

            After doing some more digging I think I've narrowed down the culprit. This issue was introduced to LTS line after Jenkins ver. 1.532.3. In 1.532.3, the maven plugin user is version 2.0.X. After 1.532.3 when I start seeing the issue, the maven plugin is moved to version 2.1. Between 2.0 and 2.1, the most likely change that caused this regression is https://issues.jenkins-ci.org/browse/JENKINS-19801 (Abort module build when maven build is aborted.)

            swolk swolk added a comment - - edited After doing some more digging I think I've narrowed down the culprit. This issue was introduced to LTS line after Jenkins ver. 1.532.3. In 1.532.3, the maven plugin user is version 2.0.X. After 1.532.3 when I start seeing the issue, the maven plugin is moved to version 2.1. Between 2.0 and 2.1, the most likely change that caused this regression is https://issues.jenkins-ci.org/browse/JENKINS-19801 (Abort module build when maven build is aborted.)
            danielbeck Daniel Beck added a comment -

            Would be helpful if we knew in which non LTS version it was introduced if it started between 1.532.3 and 1.554.1...

            danielbeck Daniel Beck added a comment - Would be helpful if we knew in which non LTS version it was introduced if it started between 1.532.3 and 1.554.1...
            swolk swolk added a comment -

            The bug was introduced in 1.545. That is the first version that used Maven Project Plugin 2.1. Looking at the code I'm very confident was introduced by changes for https://issues.jenkins-ci.org/browse/JENKINS-19801. That code assumes if modules are still building that main build was aborted, it doesn't take into account the case of parallel module building through -T option and another module failed causing the main build to terminate.

            swolk swolk added a comment - The bug was introduced in 1.545. That is the first version that used Maven Project Plugin 2.1. Looking at the code I'm very confident was introduced by changes for https://issues.jenkins-ci.org/browse/JENKINS-19801 . That code assumes if modules are still building that main build was aborted, it doesn't take into account the case of parallel module building through -T option and another module failed causing the main build to terminate.
            danielbeck Daniel Beck made changes -
            Link This issue is related to JENKINS-19801 [ JENKINS-19801 ]
            danielbeck Daniel Beck made changes -
            Component/s core [ 15593 ]
            rincewind77 Maik Ebert added a comment -

            We also suffer from this problem.

            Is there any workaround available?

            rincewind77 Maik Ebert added a comment - We also suffer from this problem. Is there any workaround available?
            ediren edwin renner added a comment -

            +1 we are suffering from this problem as well.

            ediren edwin renner added a comment - +1 we are suffering from this problem as well.
            troymills Troy Mills added a comment -

            were encountering this issue now as well..

            troymills Troy Mills added a comment - were encountering this issue now as well..
            tba T.B. Anton added a comment -

            We are encountering this issue in version 1.580.2 (Maven Project Plugin 2.8).

            tba T.B. Anton added a comment - We are encountering this issue in version 1.580.2 (Maven Project Plugin 2.8).
            michael_sawyer_veeva Michael Sawyer added a comment - Fix in Pull Request #44 .
            jglick Jesse Glick made changes -
            Labels regression

            Why is the fix not merged? We suffer from this too.

            mosesonline Slawomir Messner added a comment - Why is the fix not merged? We suffer from this too.

            We have been using our build of this plugin since June with no real issues. Sometimes a module will continue running after the build is Failed/Aborted, but it does eventually die.

            michael_sawyer_veeva Michael Sawyer added a comment - We have been using our build of this plugin since June with no real issues. Sometimes a module will continue running after the build is Failed/Aborted, but it does eventually die.
            jglick Jesse Glick added a comment -

            The PR has some comments which have not been responded to since June. There is no real plugin owner; always safer to use a freestyle project.

            jglick Jesse Glick added a comment - The PR has some comments which have not been responded to since June. There is no real plugin owner; always safer to use a freestyle project.

            Sorry, to clarify I have been using the version of the plugin found in the PR. I didn't notice a question asked on there though, so I didn't respond. I did notice you mentioned it is a slight improvement over what is currently out there, and again we haven't had any issues using it, so I would recommend pulling it in.

            P.S., It was cool seeing you at JUC jglick

            michael_sawyer_veeva Michael Sawyer added a comment - Sorry, to clarify I have been using the version of the plugin found in the PR. I didn't notice a question asked on there though, so I didn't respond. I did notice you mentioned it is a slight improvement over what is currently out there, and again we haven't had any issues using it, so I would recommend pulling it in. P.S., It was cool seeing you at JUC jglick
            liya Liya Katz added a comment -

            Experiencing the same issue with Jenkins 1.627, Maven plugin 2.12.1
            It is critical because developers are not notified when builds are aborted (usually manual action)
            If there is a fix, why is it still not merged?

            liya Liya Katz added a comment - Experiencing the same issue with Jenkins 1.627, Maven plugin 2.12.1 It is critical because developers are not notified when builds are aborted (usually manual action) If there is a fix, why is it still not merged?
            liya Liya Katz added a comment -

            Any update on this one?

            liya Liya Katz added a comment - Any update on this one?

            As a workaround, until this is fixed, you can use a groovy post-build script that changes the status of the build from ABORTED to FAILED in case it finds the error in the log.
            See a post I've wrote about it: http://www.tikalk.com/devops/JenkinsJobStatusChange

            yorammi Yoram Michaeli added a comment - As a workaround, until this is fixed, you can use a groovy post-build script that changes the status of the build from ABORTED to FAILED in case it finds the error in the log. See a post I've wrote about it: http://www.tikalk.com/devops/JenkinsJobStatusChange
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 158062 ] JNJira + In-Review [ 179715 ]
            aheritier Arnaud Héritier made changes -
            Assignee Arnaud Héritier [ aheritier ]
            aheritier Arnaud Héritier made changes -
            Status Open [ 1 ] In Progress [ 3 ]

            I just created a new PR ( https://github.com/jenkinsci/maven-plugin/pull/79 ) which is taking care of jglick comments. I hope to merge it in the next release if the review is ok

            aheritier Arnaud Héritier added a comment - I just created a new PR ( https://github.com/jenkinsci/maven-plugin/pull/79 ) which is taking care of jglick comments. I hope to merge it in the next release if the review is ok
            aheritier Arnaud Héritier made changes -
            Remote Link This issue links to "PR#79 (Web Link)" [ 14863 ]

            Fixed in >= 2.14

            aheritier Arnaud Héritier added a comment - Fixed in >= 2.14
            aheritier Arnaud Héritier made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]

            Code changed in jenkins
            User: Arnaud Heritier
            Path:
            src/main/java/hudson/maven/MavenModuleSetBuild.java
            src/test/java/hudson/maven/MavenMultiModuleTest.java
            src/test/resources/hudson/maven/maven-multimod-failure.zip
            http://jenkins-ci.org/commit/maven-plugin/2e671da7a66fc24530d82daef5c7d226414e7978
            Log:
            Fixes JENKINS-24832 : Failed maven builds using -T are showing up as Aborted (#79)

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Arnaud Heritier Path: src/main/java/hudson/maven/MavenModuleSetBuild.java src/test/java/hudson/maven/MavenMultiModuleTest.java src/test/resources/hudson/maven/maven-multimod-failure.zip http://jenkins-ci.org/commit/maven-plugin/2e671da7a66fc24530d82daef5c7d226414e7978 Log: Fixes JENKINS-24832 : Failed maven builds using -T are showing up as Aborted (#79)

            People

              aheritier Arnaud Héritier
              swolk swolk
              Votes:
              10 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: