Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28479

ProjectActions (and trend graphs) do not appear for failed projects

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • Jenkins LTS 1.596.2 / Linux

      Since junit-1.2, neither the "Latest Test Result" information nor the "Test Result Trend" plot appear on the project page _if there is not at least one build in non-failed state.

      See screenshots below for an example: after deleting unstable build #11, the abovementioned items disappear.

      The problem did not exist when the plugin was still included in Jenkins (1.576), nor does it exist in versions 1.0 and 1.1 of the plugin. I suspect that this behaviour was introduced with commit 4ba5a49 ("JENKINS-23713 Compatibility with SimpleBuildStep."), but unfortunately I cannot find the exact root cause within that commit.

      This problem is not only related to the JUnit plug-in. In general, every plug-in that has been migrated its actions to a SimpleBuildStep has this problem. The affected source code is in SimpleBuildStep.

      A test case that exposes this bug is WarningsPluginTest#should_not_skip_failed_builds_with_option_run_always

      See also the corresponding mailing list discussion.

        1. patch.txt
          3 kB
        2. withOutTrend.png
          withOutTrend.png
          60 kB
        3. withTrend.png
          withTrend.png
          81 kB

          [JENKINS-28479] ProjectActions (and trend graphs) do not appear for failed projects

          Tony Bridges added a comment -

          @Alexander +1. Conceptually, for me, this boils down to whether the job completed, and not whether it passed all tests. I have multiple records of completed runs with failed tests, and I need the trends to determine if we are getting closer to, or farther from, passing.
          Is the issue that the junit plugin is failing the job, rather than reporting faults ?

          Tony Bridges added a comment - @Alexander +1. Conceptually, for me, this boils down to whether the job completed, and not whether it passed all tests. I have multiple records of completed runs with failed tests, and I need the trends to determine if we are getting closer to, or farther from, passing. Is the issue that the junit plugin is failing the job, rather than reporting faults ?

          Daniel Beck added a comment -

          Failing tests should not fail the build. How does that happen?

          Daniel Beck added a comment - Failing tests should not fail the build. How does that happen?

          Tony Bridges added a comment -

          Looks like that might be particular to my job - just scraped the log and found that a test threw a file not found exception, which caused the job to fail. So perhaps that is my specific problem. The job never actually completed, but still showed failed tests.

          Tony Bridges added a comment - Looks like that might be particular to my job - just scraped the log and found that a test threw a file not found exception, which caused the job to fail. So perhaps that is my specific problem. The job never actually completed, but still showed failed tests.

          A Maven Job with test failures is marked as UNSTABLE (Build Success in the logs) whereas A Freestyle Job running the same project is marked a FAILED (Build Failure in the logs). Not sure if the latter is expected. In both case, surefire is reporting test failures.

          According to the [Jenkins terminology](https://wiki.jenkins-ci.org/display/JENKINS/Terminology), a build is unstable if it was built successfully and one or more publishers report it unstable. To fix this behaviour for a Freestyle project, a workaround is to add the property "maven.test.failure.ignore=true" in the "Invoke top-level Maven target" build step: The project is built successfully and the job marked as UNSTABLE.

          Allan BURDAJEWICZ added a comment - A Maven Job with test failures is marked as UNSTABLE (Build Success in the logs) whereas A Freestyle Job running the same project is marked a FAILED (Build Failure in the logs). Not sure if the latter is expected. In both case, surefire is reporting test failures. According to the [Jenkins terminology] ( https://wiki.jenkins-ci.org/display/JENKINS/Terminology ), a build is unstable if it was built successfully and one or more publishers report it unstable . To fix this behaviour for a Freestyle project, a workaround is to add the property "maven.test.failure.ignore=true" in the "Invoke top-level Maven target" build step: The project is built successfully and the job marked as UNSTABLE.

          Daniel Beck added a comment - - edited

          Right, so what happens here is that Maven by default fails the build if tests fail, and accepts an optional argument to not do that. The expectation in Jenkins is that in such a case, builds should be unstable, i.e. the build step succeeds and e.g. the JUnit plugin would mark it unstable since there are failed tests.

          Maven jobs OTOH are so magical that they set that option by default.

          Daniel Beck added a comment - - edited Right, so what happens here is that Maven by default fails the build if tests fail, and accepts an optional argument to not do that. The expectation in Jenkins is that in such a case, builds should be unstable, i.e. the build step succeeds and e.g. the JUnit plugin would mark it unstable since there are failed tests. Maven jobs OTOH are so magical that they set that option by default.

          Alexander Ost added a comment -

          Daniel, we're using a freestyle job to run tests in a custom unit test environment. If _any _of the tests fails, then the run will be marked as FAILED.

          The junit-plugin is used to display the test results and to provide convenient access to logs etc. Some of this information (trend plot, link to latest test result) are showing up only when there has ever been at least one PASSED or UNSTABLE run – and that's why I created this issue.

          Alexander Ost added a comment - Daniel, we're using a freestyle job to run tests in a custom unit test environment. If _any _of the tests fails, then the run will be marked as FAILED. The junit-plugin is used to display the test results and to provide convenient access to logs etc. Some of this information (trend plot, link to latest test result) are showing up only when there has ever been at least one PASSED or UNSTABLE run – and that's why I created this issue.

          Yafim Kazak added a comment -

          I agree with Alex. We run load tests and other test which are customers would like to fail build if failed but still see the results.
          I believe that it should be changed as in the TODO to Successful build.
          It's a wrong paradigm to get last successful job to check for actions - than to scan the job for builds with actions.
          Users would like to see what failed it - especially when we talk in the world of Jenkins as an automation server - it should provide the test results even if it failed due to bad score in tests or etc. The only other option is unstable which doesn't serve properly the meaning of successfully run but failed a critical test.

          Yafim Kazak added a comment - I agree with Alex. We run load tests and other test which are customers would like to fail build if failed but still see the results. I believe that it should be changed as in the TODO to Successful build. It's a wrong paradigm to get last successful job to check for actions - than to scan the job for builds with actions. Users would like to see what failed it - especially when we talk in the world of Jenkins as an automation server - it should provide the test results even if it failed due to bad score in tests or etc. The only other option is unstable which doesn't serve properly the meaning of successfully run but failed a critical test.

          Ulli Hafner added a comment -

          I changed the title since the problem is not related to JUnit plug-in only. Every plug-in that shows project actions has this problem, so I think we should have a fix in core. I do not see why we can't replace j.getLastSuccessfulBuild with j.getLastBuild in LastBuildActionFactory.

          Ulli Hafner added a comment - I changed the title since the problem is not related to JUnit plug-in only. Every plug-in that shows project actions has this problem, so I think we should have a fix in core. I do not see why we can't replace j.getLastSuccessfulBuild with j.getLastBuild in LastBuildActionFactory.

          Jesse Glick added a comment -

          Explained on jenkinsci-dev why that will not work. My suggestion was to amend the logic to check both getLastSuccessfulBuild and getLastCompletedBuild in case both exist but the latter is newer; collect all the LastBuildAction from each; for each LastBuildAction class, pick the newest of the two choices, if both exist; then call getProjectActions on those.

          Jesse Glick added a comment - Explained on jenkinsci-dev why that will not work. My suggestion was to amend the logic to check both getLastSuccessfulBuild and getLastCompletedBuild in case both exist but the latter is newer; collect all the LastBuildAction from each; for each LastBuildAction class, pick the newest of the two choices, if both exist; then call getProjectActions on those.

          Ulli Hafner added a comment -

          That sounds like a reasonable workaround.

          Here is a test case to reproduce the bug (just remove the Ignore tag): should_not_skip_failed_builds_with_option_run_always.

          Ulli Hafner added a comment - That sounds like a reasonable workaround. Here is a test case to reproduce the bug (just remove the Ignore tag): should_not_skip_failed_builds_with_option_run_always .

            Unassigned Unassigned
            arost Alexander Ost
            Votes:
            7 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated: