Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64299

Load statistics medium shows wrong time period

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • core
    • None
    • Jenkins 2.222.4
    • 2.342

      Load statistics chart for timespan "Medium" shows wrong time period or respectively is no more updated. 

      Charts for "Short" and "Long" are ok.
      Screenshots created Nov. 24th 2020, at 8:30. Graph for "medium timespan" shows period from 17:19 to 17:09, but I´m expecting 8:30 to 8:30.

      Medium:

      Short:

      Long:

       

          [JENKINS-64299] Load statistics medium shows wrong time period

          Kalle Niemitalo added a comment - - edited

          Added more println statements:

          import java.text.DateFormat;
          import org.jfree.data.category.DefaultCategoryDataset;
          
          MultiStageTimeSeries.TimeScale timeScale = MultiStageTimeSeries.TimeScale.MIN;
          List<MultiStageTimeSeries> series = [Jenkins.get().overallLoad.queueLength];
          
          float[][] dataPoints = new float[series.size()][];
          for (int i = 0; i < series.size(); i++)
          	dataPoints[i] = series.get(i).pick(timeScale).getHistory();
          
          int dataLength = dataPoints[0].length;
          for (float[] dataPoint : dataPoints)
          	assert dataLength == dataPoint.length;
          println("dataLength = ${dataLength}");
          
          DefaultCategoryDataset ds = new DefaultCategoryDataset();
          
          DateFormat format = timeScale.createDateFormat();
          
          Date dt = new Date(System.currentTimeMillis() - timeScale.tick * dataLength);
          for (int i = dataLength - 1; i >= 0; i--) {
          	dt = new Date(dt.getTime() + timeScale.tick);
          	String l = format.format(dt);
          	println("${dt} = ${l}");
          	for (int j = 0; j < dataPoints.length; j++)
          		ds.addValue(dataPoints[j][i], series.get(j).title.toString(), l);
          }
          
          println(ds.getColumnCount());
          println(ds.getColumnKeys());
          

          The output started with

          dataLength = 2577
          Tue Mar 22 22:17:49 EET 2022 = 22:17
          Tue Mar 22 22:18:49 EET 2022 = 22:18
          Tue Mar 22 22:19:49 EET 2022 = 22:19
          

          then continued up to

          Thu Mar 24 17:10:49 EET 2022 = 17:10
          Thu Mar 24 17:11:49 EET 2022 = 17:11
          Thu Mar 24 17:12:49 EET 2022 = 17:12
          Thu Mar 24 17:13:49 EET 2022 = 17:13
          1440
          

          followed by the column keys, of which the first was 22:17 and the last was 22:16.

          It seems to me that DefaultCategoryDataset discarded the last values added to it.

          Kalle Niemitalo added a comment - - edited Added more println statements: import java.text.DateFormat; import org.jfree.data.category.DefaultCategoryDataset; MultiStageTimeSeries.TimeScale timeScale = MultiStageTimeSeries.TimeScale.MIN; List<MultiStageTimeSeries> series = [Jenkins.get().overallLoad.queueLength]; float [][] dataPoints = new float [series.size()][]; for ( int i = 0; i < series.size(); i++) dataPoints[i] = series.get(i).pick(timeScale).getHistory(); int dataLength = dataPoints[0].length; for ( float [] dataPoint : dataPoints) assert dataLength == dataPoint.length; println( "dataLength = ${dataLength}" ); DefaultCategoryDataset ds = new DefaultCategoryDataset(); DateFormat format = timeScale.createDateFormat(); Date dt = new Date( System .currentTimeMillis() - timeScale.tick * dataLength); for ( int i = dataLength - 1; i >= 0; i--) { dt = new Date(dt.getTime() + timeScale.tick); String l = format.format(dt); println( "${dt} = ${l}" ); for ( int j = 0; j < dataPoints.length; j++) ds.addValue(dataPoints[j][i], series.get(j).title.toString(), l); } println(ds.getColumnCount()); println(ds.getColumnKeys()); The output started with dataLength = 2577 Tue Mar 22 22:17:49 EET 2022 = 22:17 Tue Mar 22 22:18:49 EET 2022 = 22:18 Tue Mar 22 22:19:49 EET 2022 = 22:19 then continued up to Thu Mar 24 17:10:49 EET 2022 = 17:10 Thu Mar 24 17:11:49 EET 2022 = 17:11 Thu Mar 24 17:12:49 EET 2022 = 17:12 Thu Mar 24 17:13:49 EET 2022 = 17:13 1440 followed by the column keys, of which the first was 22:17 and the last was 22:16. It seems to me that DefaultCategoryDataset discarded the last values added to it.

          Kalle Niemitalo added a comment - - edited

          Oh maybe the problem is that SimpleDateFormat("HH:mm") generates identical column keys for the consecutive dates. Then jfreechart assumes that they mean the same column and doesn't add more columns to the graph.

          Kalle Niemitalo added a comment - - edited Oh maybe the problem is that SimpleDateFormat("HH:mm") generates identical column keys for the consecutive dates. Then jfreechart assumes that they mean the same column and doesn't add more columns to the graph.

          Kalle Niemitalo added a comment - - edited

          core/src/main/java/hudson/model/MultiStageTimeSeries.java has a comment saying only one day of data should be kept here:

              /**
               * Updated every 1 min. Keep data up to 1 day.
               */
              @Exported
              public final TimeSeries min;
          

          but it actually stores two days:

                  this.min = new TimeSeries(initialValue, decay, (int) TimeUnit.DAYS.toMinutes(2));
          

          If this were changed to toMinutes(1) like the comment says, then I think that would fix the bug, by not generating duplicate column keys.

          Caused by commit 0f6a398b0f397196708770ea473f7436e4bbf560 "Add more data points to MultiStageTimeSeries" in https://github.com/jenkinsci/jenkins/pull/4341, so jenkins-2.204 is the first version with this bug.

          Kalle Niemitalo added a comment - - edited core/src/main/java/hudson/model/MultiStageTimeSeries.java has a comment saying only one day of data should be kept here: /** * Updated every 1 min. Keep data up to 1 day. */ @Exported public final TimeSeries min; but it actually stores two days: this .min = new TimeSeries(initialValue, decay, ( int ) TimeUnit.DAYS.toMinutes(2)); If this were changed to toMinutes(1) like the comment says, then I think that would fix the bug, by not generating duplicate column keys. Caused by commit 0f6a398b0f397196708770ea473f7436e4bbf560 "Add more data points to MultiStageTimeSeries" in https://github.com/jenkinsci/jenkins/pull/4341 , so jenkins-2.204 is the first version with this bug.

          Alternatively, the date format could be changed to include the day number, or the day of week.

          Kalle Niemitalo added a comment - Alternatively, the date format could be changed to include the day number, or the day of week.

          Daniel Beck added a comment - - edited

          Interesting, I had a suspicion this was my change but I didn't see why. This makes sense.

          The original reported issue would have been present before that though, because first and last entry would have had the same label even for just 1 day of data, correct?

          Daniel Beck added a comment - - edited Interesting, I had a suspicion this was my change but I didn't see why. This makes sense. The original reported issue would have been present before that though, because first and last entry would have had the same label even for just 1 day of data, correct?

          Kalle Niemitalo added a comment - - edited

          No, dataLength was at most 1440 originally, and 1440 distinct column keys were generated; one per minute.

          Another possible fix might be to create custom objects as column keys. These would implement Comparable by comparing the underlying Date values, and override toString(). That way, the correctness would not depend on the date format.

          Or use the Date values as column keys and override AbstractCategoryItemLabelGenerator.generateColumnLabel, but that might require larger changes in Jenkins.

          Kalle Niemitalo added a comment - - edited No, dataLength was at most 1440 originally, and 1440 distinct column keys were generated; one per minute. Another possible fix might be to create custom objects as column keys. These would implement Comparable by comparing the underlying Date values, and override toString(). That way, the correctness would not depend on the date format. Or use the Date values as column keys and override AbstractCategoryItemLabelGenerator.generateColumnLabel , but that might require larger changes in Jenkins.

          Daniel Beck added a comment -

          Fix attempt in https://github.com/jenkinsci/jenkins/pull/6402

          While I have not seen the original reported behavior, the data does not extend past 1 day at the moment even in a "good" situation; so the PR should at a minimum address that, if the hypothesis by kon is correct.

          Daniel Beck added a comment - Fix attempt in https://github.com/jenkinsci/jenkins/pull/6402 While I have not seen the original reported behavior, the data does not extend past 1 day at the moment even in a "good" situation; so the PR should at a minimum address that, if the hypothesis by kon is correct.

          At https://ci.jenkins.io/load-statistics?type=min, the graph looks OK, but it displays only one day worth of data rather than two. I think this is what happens when the Jenkins controller has been running for two days or longer. The graph breaks in a worse way only when Jenkins has been running for longer than one day, but less than two days.

          Kalle Niemitalo added a comment - At https://ci.jenkins.io/load-statistics?type=min , the graph looks OK, but it displays only one day worth of data rather than two. I think this is what happens when the Jenkins controller has been running for two days or longer. The graph breaks in a worse way only when Jenkins has been running for longer than one day, but less than two days.

          Screenshots created Nov. 24th 2020, at 8:30. Graph for "medium timespan" shows period from 17:19 to 17:09, but I´m expecting 8:30 to 8:30.

          Medium:

          Strange that this graph has a discontinuity near 09:23 rather than 08:30.

          Kalle Niemitalo added a comment - Screenshots created Nov. 24th 2020, at 8:30 . Graph for "medium timespan" shows period from 17:19 to 17:09, but I´m expecting 8:30 to 8:30. Medium: Strange that this graph has a discontinuity near 09:23 rather than 08:30.

          Daniel Beck added a comment -

          Strange that this graph has a discontinuity near 09:23 rather than 08:30.

          TZ? My local Jenkins is currently 2 hours off (latest label is "Sun 13:07" while it's 15:24), which corresponds to the (new) CEST offset from UTC.

          Daniel Beck added a comment - Strange that this graph has a discontinuity near 09:23 rather than 08:30. TZ? My local Jenkins is currently 2 hours off (latest label is "Sun 13:07" while it's 15:24), which corresponds to the (new) CEST offset from UTC.

            danielbeck Daniel Beck
            hjhafner Hans-Juergen Hafner
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: