Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20438

Make it an option to show failed builds in trend graph

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      There was an issue 'JENKINS-16832 Do not show failed builds in trend graph' which was fixed. Now, failed builds are not shown in trend graph.

      We have test dashboard with trend graph. We run nightly regression each day. If build is failed then we don't see it in the graph and we can easily mislead that test build was fine, but it wasn't.

      Is it possible to make an optional to show failed and aborted builds in trend graph? This option can be disabled by default.

        Attachments

          Issue Links

            Activity

            Hide
            nullin Nalin Makar added a comment - - edited

            Why does the build fail? If this is a maven build, use "-Dmaven.test.failure.ignore=true" to ensure that the build is not marked as failed even on test failures. This way you'll correctly see that the build is unstable.

            Show
            nullin Nalin Makar added a comment - - edited Why does the build fail? If this is a maven build, use "-Dmaven.test.failure.ignore=true" to ensure that the build is not marked as failed even on test failures. This way you'll correctly see that the build is unstable.
            Hide
            piwko Ivan Sas added a comment -

            We use gradle build and have setIgnoreFailures set to true. We have test dashboard with aggregated embedded of multiple test results graphs for different products.

            We have few thousands tests and it sometimes occurs that build is not created, for any reason.

            For example some external xml schemas were not available during the build time, or build causes out of memory, or someone committed code which is not compiling. Another example which occurs last time. Someone has committed code which didn't close files. Tests opened so many files. We are doing some conversion of testNG xml reports at the end of test suite and then pass it to jenkins testng plugin. There were opened so many files, which causes that it was unable to create test XML report and the build was failed.

            The purpose of this change is to know the state of tests even if build was not prepared for any reason. If someone asked about nightly tests result, I can say from the graph that we have no test result, because something goes wrong with the build.

            Show
            piwko Ivan Sas added a comment - We use gradle build and have setIgnoreFailures set to true. We have test dashboard with aggregated embedded of multiple test results graphs for different products. We have few thousands tests and it sometimes occurs that build is not created, for any reason. For example some external xml schemas were not available during the build time, or build causes out of memory, or someone committed code which is not compiling. Another example which occurs last time. Someone has committed code which didn't close files. Tests opened so many files. We are doing some conversion of testNG xml reports at the end of test suite and then pass it to jenkins testng plugin. There were opened so many files, which causes that it was unable to create test XML report and the build was failed. The purpose of this change is to know the state of tests even if build was not prepared for any reason. If someone asked about nightly tests result, I can say from the graph that we have no test result, because something goes wrong with the build.
            Hide
            ggb667 Greg Bishop added a comment -

            I'm sort of shocked this doesn't exist already.
            There's really no way to know that 102 tests failed out of 10,222 yesterday, and today 105 are failing?

            Show
            ggb667 Greg Bishop added a comment - I'm sort of shocked this doesn't exist already. There's really no way to know that 102 tests failed out of 10,222 yesterday, and today 105 are failing?
            Hide
            dpstewart Doug Stewart added a comment -

            In my test environment I set up the Jenkins jobs to run my tests using a windows batch file. Any test failures results in a failed build and the test results are not then included in the graph. I can work around this by using EXIT /B 0 in my batch file but this then does not indicate whether there were any failures for this particular test run. I was relying on a failed test run/build to send me notification which no longer happens. I feel it would be better if including the results in the graph or not was set up as an available option.

            Show
            dpstewart Doug Stewart added a comment - In my test environment I set up the Jenkins jobs to run my tests using a windows batch file. Any test failures results in a failed build and the test results are not then included in the graph. I can work around this by using EXIT /B 0 in my batch file but this then does not indicate whether there were any failures for this particular test run. I was relying on a failed test run/build to send me notification which no longer happens. I feel it would be better if including the results in the graph or not was set up as an available option.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Nalin Makar
            Path:
            README
            src/main/java/hudson/plugins/testng/Publisher.java
            src/main/java/hudson/plugins/testng/TestNGProjectAction.java
            src/main/resources/hudson/plugins/testng/Publisher/config.jelly
            src/main/resources/hudson/plugins/testng/Publisher/help-showFailedBuilds.html
            src/test/java/hudson/plugins/testng/PublisherTest.java
            src/test/java/hudson/plugins/testng/TestNGProjectActionTest.java
            src/test/java/hudson/plugins/testng/TestNGTestResultBuildActionTest.java
            src/test/java/hudson/plugins/testng/results/ClassResultTest.java
            src/test/java/hudson/plugins/testng/results/MethodResultTest.java
            src/test/java/hudson/plugins/testng/results/PackageResultTest.java
            http://jenkins-ci.org/commit/testng-plugin-plugin/f984cfafde186d3e1f3a779fccb8a2c61a20007d
            Log:
            Added: JENKINS-20438 - Make it an option to show failed builds in trend graph

            Compare: https://github.com/jenkinsci/testng-plugin-plugin/compare/b03c0ad77827...f984cfafde18

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nalin Makar Path: README src/main/java/hudson/plugins/testng/Publisher.java src/main/java/hudson/plugins/testng/TestNGProjectAction.java src/main/resources/hudson/plugins/testng/Publisher/config.jelly src/main/resources/hudson/plugins/testng/Publisher/help-showFailedBuilds.html src/test/java/hudson/plugins/testng/PublisherTest.java src/test/java/hudson/plugins/testng/TestNGProjectActionTest.java src/test/java/hudson/plugins/testng/TestNGTestResultBuildActionTest.java src/test/java/hudson/plugins/testng/results/ClassResultTest.java src/test/java/hudson/plugins/testng/results/MethodResultTest.java src/test/java/hudson/plugins/testng/results/PackageResultTest.java http://jenkins-ci.org/commit/testng-plugin-plugin/f984cfafde186d3e1f3a779fccb8a2c61a20007d Log: Added: JENKINS-20438 - Make it an option to show failed builds in trend graph Compare: https://github.com/jenkinsci/testng-plugin-plugin/compare/b03c0ad77827...f984cfafde18
            Hide
            nullin Nalin Makar added a comment -

            Fixed. option will be available in next release.

            Show
            nullin Nalin Makar added a comment - Fixed. option will be available in next release.
            Hide
            ankush ankush goyal added a comment -

            Nalin Makar i have toggle ON the option    "Show failed builds in trend graph", still not able to see the failed build in TestNG result Trend graph

            Show
            ankush ankush goyal added a comment - Nalin Makar i have toggle ON the option    "Show failed builds in trend graph", still not able to see the failed build in TestNG result Trend graph

              People

              Assignee:
              nullin Nalin Makar
              Reporter:
              piwko Ivan Sas
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: