-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
Jenkins 2.222.4
-
Powered by SuggestiMate -
2.342
Load statistics chart for timespan "Medium" shows wrong time period or respectively is no more updated.
Charts for "Short" and "Long" are ok.
Screenshots created Nov. 24th 2020, at 8:30. Graph for "medium timespan" shows period from 17:19 to 17:09, but I´m expecting 8:30 to 8:30.
Medium:
Long:
[JENKINS-64299] Load statistics medium shows wrong time period
Yes, I tried reloading the page, I closed and reopened the browser and I restarted my PC.
Other users of our Jenkins had same issue.
After restart of Jenkins statistics are ok again.
Could it be that a process got stuck?
Also started seeing this today, reloading page ignoring cache does not help.
I'm seeing this again, on Jenkins 2.332.1. It cannot be a caching issue, because the graph is still wrong even if I edit the width parameter in the URL of the graph. The bug is affecting only type=min; the graphs for type=sec10 and type=hour look OK.
kon Are you comfortable enough with the Script Console to dig into the recorded data to identify any anomalies?
The local time was 16:32. I ran in the Script Console:
println(MultiStageTimeSeries.createTrendChart(MultiStageTimeSeries.TimeScale.SEC10, Jenkins.get().overallLoad.queueLength).dataset.getColumnKeys()) println(MultiStageTimeSeries.createTrendChart(MultiStageTimeSeries.TimeScale.MIN, Jenkins.get().overallLoad.queueLength).dataset.getColumnKeys()) println(MultiStageTimeSeries.createTrendChart(MultiStageTimeSeries.TimeScale.HOUR, Jenkins.get().overallLoad.queueLength).dataset.getColumnKeys())
The first and last values in each list were:
- SEC10: [10:32:39, 10:32:49, 10:32:59, 10:33:09, 10:33:19, 10:33:29, 10:33:39, 10:33:49, 10:33:59, 10:34:09, 10:34:19, 10:34:29, …, 16:30:39, 16:30:49, 16:30:59, 16:31:09, 16:31:19, 16:31:29, 16:31:39, 16:31:49, 16:31:59, 16:32:09, 16:32:19, 16:32:29]
- MIN: [22:17, 22:18, 22:19, 22:20, 22:21, 22:22, 22:23, 22:24, 22:25, 22:26, 22:27, 22:28, 22:29, 22:30, 22:31, 22:32, …, 21:59, 22:00, 22:01, 22:02, 22:03, 22:04, 22:05, 22:06, 22:07, 22:08, 22:09, 22:10, 22:11, 22:12, 22:13, 22:14, 22:15, 22:16]
- HOUR: [maalisk./22 22, maalisk./22 23, maalisk./23 00, maalisk./23 01, maalisk./23 02, maalisk./23 03, maalisk./23 04, maalisk./23 05, maalisk./23 06, maalisk./23 07, maalisk./23 08, maalisk./23 09, maalisk./23 10, maalisk./23 11, maalisk./23 12, maalisk./23 13, maalisk./23 14, maalisk./23 15, maalisk./23 16, maalisk./23 17, maalisk./23 18, maalisk./23 19, maalisk./23 20, maalisk./23 21, maalisk./23 22, maalisk./23 23, maalisk./24 00, maalisk./24 01, maalisk./24 02, maalisk./24 03, maalisk./24 04, maalisk./24 05, maalisk./24 06, maalisk./24 07, maalisk./24 08, maalisk./24 09, maalisk./24 10, maalisk./24 11, maalisk./24 12, maalisk./24 13, maalisk./24 14, maalisk./24 15, maalisk./24 16]
The last column key in the MIN dataset was 22:16 even though it should have been 16:32.
What is the output of:
println Jenkins.get().overallLoad.queueLength.sec10.history println Jenkins.get().overallLoad.queueLength.sec10.decay println Jenkins.get().overallLoad.queueLength.sec10.historySize println Jenkins.get().overallLoad.queueLength.min.history println Jenkins.get().overallLoad.queueLength.min.decay println Jenkins.get().overallLoad.queueLength.min.historySize println Jenkins.get().overallLoad.queueLength.hour.history println Jenkins.get().overallLoad.queueLength.hour.decay println Jenkins.get().overallLoad.queueLength.hour.historySize
Imitating MultiStageTimeSeries.TrendChart.createDataset():
import java.text.DateFormat; import org.jfree.data.category.DefaultCategoryDataset; MultiStageTimeSeries.TimeScale timeScale = MultiStageTimeSeries.TimeScale.MIN; List<MultiStageTimeSeries> series = [Jenkins.get().overallLoad.queueLength]; float[][] dataPoints = new float[series.size()][]; for (int i = 0; i < series.size(); i++) dataPoints[i] = series.get(i).pick(timeScale).getHistory(); int dataLength = dataPoints[0].length; for (float[] dataPoint : dataPoints) assert dataLength == dataPoint.length; DefaultCategoryDataset ds = new DefaultCategoryDataset(); DateFormat format = timeScale.createDateFormat(); Date dt = new Date(System.currentTimeMillis() - timeScale.tick * dataLength); for (int i = dataLength - 1; i >= 0; i--) { dt = new Date(dt.getTime() + timeScale.tick); String l = format.format(dt); for (int j = 0; j < dataPoints.length; j++) ds.addValue(dataPoints[j][i], series.get(j).title.toString(), l); } println(ds.getColumnKeys());
likewise outputs [22:17, 22:18, 22:19, 22:20, 22:21, 22:22, 22:23, 22:24, 22:25, 22:26, 22:27, 22:28, 22:29, 22:30, …, 22:01, 22:02, 22:03, 22:04, 22:05, 22:06, 22:07, 22:08, 22:09, 22:10, 22:11, 22:12, 22:13, 22:14, 22:15, 22:16]. Notably, the last column key is still 22:16 even though 24 minutes have passed.
I'll try adding more println statements.
println Jenkins.get().overallLoad.queueLength.sec10.history println Jenkins.get().overallLoad.queueLength.sec10.decay println Jenkins.get().overallLoad.queueLength.sec10.historySize println Jenkins.get().overallLoad.queueLength.min.history println Jenkins.get().overallLoad.queueLength.min.decay println Jenkins.get().overallLoad.queueLength.min.historySize println Jenkins.get().overallLoad.queueLength.hour.history println Jenkins.get().overallLoad.queueLength.hour.decay println Jenkins.get().overallLoad.queueLength.hour.historySize
The complete result exceeds the Jira limit of 32767 characters. JENKINS-64299 queueLength.txt
Added more println statements:
import java.text.DateFormat; import org.jfree.data.category.DefaultCategoryDataset; MultiStageTimeSeries.TimeScale timeScale = MultiStageTimeSeries.TimeScale.MIN; List<MultiStageTimeSeries> series = [Jenkins.get().overallLoad.queueLength]; float[][] dataPoints = new float[series.size()][]; for (int i = 0; i < series.size(); i++) dataPoints[i] = series.get(i).pick(timeScale).getHistory(); int dataLength = dataPoints[0].length; for (float[] dataPoint : dataPoints) assert dataLength == dataPoint.length; println("dataLength = ${dataLength}"); DefaultCategoryDataset ds = new DefaultCategoryDataset(); DateFormat format = timeScale.createDateFormat(); Date dt = new Date(System.currentTimeMillis() - timeScale.tick * dataLength); for (int i = dataLength - 1; i >= 0; i--) { dt = new Date(dt.getTime() + timeScale.tick); String l = format.format(dt); println("${dt} = ${l}"); for (int j = 0; j < dataPoints.length; j++) ds.addValue(dataPoints[j][i], series.get(j).title.toString(), l); } println(ds.getColumnCount()); println(ds.getColumnKeys());
The output started with
dataLength = 2577 Tue Mar 22 22:17:49 EET 2022 = 22:17 Tue Mar 22 22:18:49 EET 2022 = 22:18 Tue Mar 22 22:19:49 EET 2022 = 22:19
then continued up to
Thu Mar 24 17:10:49 EET 2022 = 17:10 Thu Mar 24 17:11:49 EET 2022 = 17:11 Thu Mar 24 17:12:49 EET 2022 = 17:12 Thu Mar 24 17:13:49 EET 2022 = 17:13 1440
followed by the column keys, of which the first was 22:17 and the last was 22:16.
It seems to me that DefaultCategoryDataset discarded the last values added to it.
Oh maybe the problem is that SimpleDateFormat("HH:mm") generates identical column keys for the consecutive dates. Then jfreechart assumes that they mean the same column and doesn't add more columns to the graph.
core/src/main/java/hudson/model/MultiStageTimeSeries.java has a comment saying only one day of data should be kept here:
/** * Updated every 1 min. Keep data up to 1 day. */ @Exported public final TimeSeries min;
but it actually stores two days:
this.min = new TimeSeries(initialValue, decay, (int) TimeUnit.DAYS.toMinutes(2));
If this were changed to toMinutes(1) like the comment says, then I think that would fix the bug, by not generating duplicate column keys.
Caused by commit 0f6a398b0f397196708770ea473f7436e4bbf560 "Add more data points to MultiStageTimeSeries" in https://github.com/jenkinsci/jenkins/pull/4341, so jenkins-2.204 is the first version with this bug.
Alternatively, the date format could be changed to include the day number, or the day of week.
Interesting, I had a suspicion this was my change but I didn't see why. This makes sense.
The original reported issue would have been present before that though, because first and last entry would have had the same label even for just 1 day of data, correct?
No, dataLength was at most 1440 originally, and 1440 distinct column keys were generated; one per minute.
Another possible fix might be to create custom objects as column keys. These would implement Comparable by comparing the underlying Date values, and override toString(). That way, the correctness would not depend on the date format.
Or use the Date values as column keys and override AbstractCategoryItemLabelGenerator.generateColumnLabel, but that might require larger changes in Jenkins.
Fix attempt in https://github.com/jenkinsci/jenkins/pull/6402
While I have not seen the original reported behavior, the data does not extend past 1 day at the moment even in a "good" situation; so the PR should at a minimum address that, if the hypothesis by kon is correct.
At https://ci.jenkins.io/load-statistics?type=min, the graph looks OK, but it displays only one day worth of data rather than two. I think this is what happens when the Jenkins controller has been running for two days or longer. The graph breaks in a worse way only when Jenkins has been running for longer than one day, but less than two days.
Screenshots created Nov. 24th 2020, at 8:30. Graph for "medium timespan" shows period from 17:19 to 17:09, but I´m expecting 8:30 to 8:30.
Medium:
Strange that this graph has a discontinuity near 09:23 rather than 08:30.
Strange that this graph has a discontinuity near 09:23 rather than 08:30.
TZ? My local Jenkins is currently 2 hours off (latest label is "Sun 13:07" while it's 15:24), which corresponds to the (new) CEST offset from UTC.
I've never seen this. Are you sure this isn't just a caching issue? Have you tried cache-bypassing page reloads (Shift-F5 or similar)?