Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4378

Maven2 test failures not resulting in build failure, or build unstable.

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Minor Minor
    • maven-plugin
    • None
    • Platform: Macintosh, OS: Mac OS X

      I have a Maven2 project in Hudson using the Maven2 project type.

      When the project builds, there is a unit test that fails (SureFire), but the
      project build is not marked as unstable, nor does it fail.

      If I set the "maven.test.failure.ignore=false" property, the project build will
      fail, but the failed test result is not shown.

      I ran hudson manually in the terminal, but did not see any errors.

          [JENKINS-4378] Maven2 test failures not resulting in build failure, or build unstable.

          Andrew Bayer added a comment -

          This is a multi-module project, yes? Can you drill down to the module with test
          failures and see if it's marked as unstable?

          Andrew Bayer added a comment - This is a multi-module project, yes? Can you drill down to the module with test failures and see if it's marked as unstable?

          elberry added a comment -

          That's correct, it's a multi-module project, and no, if I drill down the modules
          in Hudson, they all show they passed successfully. None of the modules show that
          the tests failed, and are marked with the yellow ball.

          elberry added a comment - That's correct, it's a multi-module project, and no, if I drill down the modules in Hudson, they all show they passed successfully. None of the modules show that the tests failed, and are marked with the yellow ball.

          Alan Harder added a comment -

          Are you still able to see this problem in a recent Hudson release?
          If you can attach a simple pom+src bundle we can use to reproduce the problem it would help, thanks.

          Alan Harder added a comment - Are you still able to see this problem in a recent Hudson release? If you can attach a simple pom+src bundle we can use to reproduce the problem it would help, thanks.

          Alan Harder added a comment -

          Reopen if you still see this issue on a recent Hudson release, thanks.

          Alan Harder added a comment - Reopen if you still see this issue on a recent Hudson release, thanks.

          flottis added a comment -

          Hi,

          We are experiencing the exact same problems as mentioned above. I just installed the latest stable release (1.386) but faild junit tests does not mark the project as unstable. Some project setup facts:

          • Maven2
          • JUnit
          • Surefire
          • Multi-module

          Let me know if you need src+pom to reproduce this and I'll throw something together.

          flottis added a comment - Hi, We are experiencing the exact same problems as mentioned above. I just installed the latest stable release (1.386) but faild junit tests does not mark the project as unstable. Some project setup facts: Maven2 JUnit Surefire Multi-module Let me know if you need src+pom to reproduce this and I'll throw something together.

          Alan Harder added a comment -

          [src+pom] yes, please..

          Alan Harder added a comment - [src+pom] yes, please..

          flottis added a comment -

          I'm sure I added a comment once I solved this problem but I cant find it. Anyway this problem is solved for me and it was our own fault. We had disabled the XML report generation for the surefire tests so hudson could not pick up on the test results. Once we enabled the XML reports everything worked as expected.

          As far as I'm concerned you can close this issue.

          flottis added a comment - I'm sure I added a comment once I solved this problem but I cant find it. Anyway this problem is solved for me and it was our own fault. We had disabled the XML report generation for the surefire tests so hudson could not pick up on the test results. Once we enabled the XML reports everything worked as expected. As far as I'm concerned you can close this issue.

          I have a requirement where i want to fail the build if the number of tests failed is more than a defined threshold so setting this maven.test.failure.ignore=false will not help as i dont want to always fail the build but only based on a defined threshold.

          I have added System.exit(1) to fail the build in jenkins. It works in Free style project of jenkins but in Maven type project it does not mark the build as failure with System.exit. However when i tried running the tests in Post build step of Maven Type of Project it fails the build but same does not fail when run tests in Build Step. Please advise.

          Surinder Madhok added a comment - I have a requirement where i want to fail the build if the number of tests failed is more than a defined threshold so setting this maven.test.failure.ignore=false will not help as i dont want to always fail the build but only based on a defined threshold. I have added System.exit(1) to fail the build in jenkins. It works in Free style project of jenkins but in Maven type project it does not mark the build as failure with System.exit. However when i tried running the tests in Post build step of Maven Type of Project it fails the build but same does not fail when run tests in Build Step. Please advise.

            Unassigned Unassigned
            elberry elberry
            Votes:
            2 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: