Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33952

"Test Statistics Grid" shows all in "0" and "0%" including total as 0 tests when build never passed

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Reopened (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: dashboard-view-plugin
    • Labels:
      None
    • Environment:
      plugin version 2.9.7
      Windows Server 2012
      jenkins version 1.623
      Browsers: all
      Java 1.8
    • Similar Issues:

      Description

      Hi,

      There is a problem in the latest release:

      If you have job that history is kept let say for 14 days and it was never been green the "Test Statistics Grid" shows all in "0" and "0%" including total as 0 tests

      PS: here is a screenshot with calculations to confirm the bug - http://screencast.com/t/1QkzO8Ggy - the stats are completely wrong

        Attachments

          Activity

          Hide
          vandyev Evan Van Dyke added a comment -
          Show
          vandyev Evan Van Dyke added a comment - Addressed in pull request https://github.com/jenkinsci/dashboard-view-plugin/pull/42
          Hide
          vandyev Evan Van Dyke added a comment -

          Resolved in 2.9.8

          Show
          vandyev Evan Van Dyke added a comment - Resolved in 2.9.8
          Hide
          vandyev Evan Van Dyke added a comment -

          2.9.8

          Show
          vandyev Evan Van Dyke added a comment - 2.9.8
          Hide
          ndinev Neven DInev added a comment -

          I tested again with version 2.9.9
          Unfortunately the problem still exists - Failed tests form test sets that never passed are not included in the chart
          http://screencast.com/t/orRVRGU53

          Show
          ndinev Neven DInev added a comment - I tested again with version 2.9.9 Unfortunately the problem still exists - Failed tests form test sets that never passed are not included in the chart http://screencast.com/t/orRVRGU53
          Hide
          vandyev Evan Van Dyke added a comment -

          What plugin are you using to run/tabulate the test statistics? The Dashboard view is pretty dumb; it just reports whatever the test result actions from the plugin report. I can try and confirm/diagnose, but I'd like to make sure I replicate your environment.

          Show
          vandyev Evan Van Dyke added a comment - What plugin are you using to run/tabulate the test statistics? The Dashboard view is pretty dumb; it just reports whatever the test result actions from the plugin report. I can try and confirm/diagnose, but I'd like to make sure I replicate your environment.
          Hide
          ndinev Neven DInev added a comment -

          Not sure what you are asking, but I am using simple dashboard view with Test Statistics Chart and Test Trend Chart
          Whenever there is a NA in "Last Success" column the result of this test set is ignored in Test Statistics Chart.

          However it is important to note that Test Trend Chart (seems to) show correctly the number of failing test. This is strange since I would expect that trend chart uses same data as pie chart but shows them over time.

          Hope this helps

          Show
          ndinev Neven DInev added a comment - Not sure what you are asking, but I am using simple dashboard view with Test Statistics Chart and Test Trend Chart Whenever there is a NA in "Last Success" column the result of this test set is ignored in Test Statistics Chart. However it is important to note that Test Trend Chart (seems to) show correctly the number of failing test. This is strange since I would expect that trend chart uses same data as pie chart but shows them over time. Hope this helps
          Hide
          darthopto Curtis Salisbury added a comment -

          I am having the same problem with the Test Statistics Grid. The numbers for one job have never populated and the others seem to not update.

          Show
          darthopto Curtis Salisbury added a comment - I am having the same problem with the Test Statistics Grid. The numbers for one job have never populated and the others seem to not update.
          Hide
          vandyev Evan Van Dyke added a comment - - edited

          Neven,

          Definitely agree that it looks like something is wonky. Can you throw the "Test Statistics Grid" up on your dashboard, and show me the result of that? It shows all the individual jobs being tallied up, and that might help diagnose the issue. I don't see anywhere in the code where there should be an issue specific to never-passed jobs, so I'd like to try and get more data.

          UPDATE: I went through your latest screenshot, and it doesn't look like the "successful" tests for those two jobs are being included in the chart either. Is there anything special about those jobs? What plugin is publishing those test results?

          Show
          vandyev Evan Van Dyke added a comment - - edited Neven, Definitely agree that it looks like something is wonky. Can you throw the "Test Statistics Grid" up on your dashboard, and show me the result of that? It shows all the individual jobs being tallied up, and that might help diagnose the issue. I don't see anywhere in the code where there should be an issue specific to never-passed jobs, so I'd like to try and get more data. UPDATE: I went through your latest screenshot, and it doesn't look like the "successful" tests for those two jobs are being included in the chart either. Is there anything special about those jobs? What plugin is publishing those test results?
          Hide
          vandyev Evan Van Dyke added a comment -

          Curtis,

          Regarding the Grid, what type of tests are being published? I fixed an issue with the TAP plugin where it wasn't publishing its results in a way that the grid liked. I'd recommend opening a separate defect report for that one, tied to whatever test tool is trying to publish results. You can reference https://issues.jenkins-ci.org/browse/JENKINS-22016 which was the fix for the TAP plugin.

          Show
          vandyev Evan Van Dyke added a comment - Curtis, Regarding the Grid, what type of tests are being published? I fixed an issue with the TAP plugin where it wasn't publishing its results in a way that the grid liked. I'd recommend opening a separate defect report for that one, tied to whatever test tool is trying to publish results. You can reference https://issues.jenkins-ci.org/browse/JENKINS-22016 which was the fix for the TAP plugin.
          Hide
          jdemelo Jason DeMelo added a comment - - edited

          I'm seeing the same problem. I think it stems from the fact that TestUtil.summarizeJob() calls

          job.getAction(TestResultProjectAction.class)

          In my debugging, job.getAction() leads to the following calls:

          job.getAllActions()
            - Actionable.createFor()
              - SimpleBuildStep.LastBuildActionFactory.createFor()
          

          Inside LastBuildActionFactory.createFor(), you can see that project actions will only be added to the collection returned if getLastSuccessfulBuild() is not null. Because of this, the dashboard's calls to job.getAction(TestResultProjectAction.class) return null and a blank test result is added to the summary.

          My hunch is that in general, only showing information from the last successful build isn't what people are looking for. In my case, I always want to see information from the last build that produced a test result, even if that build failed.

          You might be able to borrow from here, and call it here, instead of job.getAction(TestResultProjectAction.class).

          Show
          jdemelo Jason DeMelo added a comment - - edited I'm seeing the same problem. I think it stems from the fact that TestUtil.summarizeJob() calls job.getAction(TestResultProjectAction.class) In my debugging, job.getAction() leads to the following calls: job.getAllActions() - Actionable.createFor() - SimpleBuildStep.LastBuildActionFactory.createFor() Inside LastBuildActionFactory.createFor(), you can see that project actions will only be added to the collection returned if getLastSuccessfulBuild() is not null . Because of this, the dashboard's calls to job.getAction(TestResultProjectAction.class) return null and a blank test result is added to the summary. My hunch is that in general, only showing information from the last successful build isn't what people are looking for. In my case, I always want to see information from the last build that produced a test result, even if that build failed. You might be able to borrow from here , and call it here , instead of job.getAction(TestResultProjectAction.class).
          Hide
          neel_narute Neel Narute added a comment -

          Hi,

          I'm also facing the same issue. Do we have any update on this?

          "Test Statistics Grid" showing '0' for failed test cases.

           

          Regards,

          Neel

          Show
          neel_narute Neel Narute added a comment - Hi, I'm also facing the same issue. Do we have any update on this? "Test Statistics Grid" showing '0' for failed test cases.   Regards, Neel
          Hide
          mekhdievzr Zaur Mekhdiev added a comment -

          Good day, guys! 

           

          Same issue

          Jenkins ver - 2.150.1

          Plugin ver - 2.10

          Show
          mekhdievzr Zaur Mekhdiev added a comment - Good day, guys!    Same issue Jenkins ver - 2.150.1 Plugin ver - 2.10
          Hide
          monsam Samrat Ghosh added a comment -

          any update on this issue?

          Show
          monsam Samrat Ghosh added a comment - any update on this issue?

            People

            Assignee:
            tgr Tobias Gruetzmacher
            Reporter:
            ndinev Neven DInev
            Votes:
            4 Vote for this issue
            Watchers:
            8 Start watching this issue

              Dates

              Created:
              Updated: