Details
-
Type:
Bug
-
Status: Closed (View Workflow)
-
Priority:
Major
-
Resolution: Fixed
-
Component/s: dashboard-view-plugin
-
Labels:
-
Environment:Jenkins 1.598, Dashboard View 2.9.4
-
Similar Issues:
Description
The Job Statistics portlet takes a long time to initially load with 100+ jobs and 1000+ builds each. Thread dump shows the stack trace is mostly spent in RunList.size() which is being called in StatJobs.HealthStatus.getHealthStatus(). This forces every build to be loaded from disk when, looking at the code, all it needs to do is check if the list is empty.
"Handling GET /view/ServersDashboard/ from 192.168.70.40 : RequestHandlerThread10 View/index.jelly Dashboard/main.jelly StatJobs/portlet.jelly StatJobs/statjobs.jelly" Id=59 Group=main RUNNABLE
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:272)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
- locked java.io.BufferedInputStream@7bb58e96
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at java.io.PushbackInputStream.read(PushbackInputStream.java:139)
at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.getHeader(XmlHeaderAwareReader.java:79)
at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.<init>(XmlHeaderAwareReader.java:61)
at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65)
at hudson.XmlFile.unmarshal(XmlFile.java:163)
at hudson.model.Run.reload(Run.java:321)
at hudson.model.Run.<init>(Run.java:309)
at hudson.model.AbstractBuild.<init>(AbstractBuild.java:178)
at hudson.model.Build.<init>(Build.java:103)
at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:38)
at sun.reflect.GeneratedConstructorAccessor70.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at jenkins.model.lazy.LazyBuildMixIn.loadBuild(LazyBuildMixIn.java:158)
at jenkins.model.lazy.LazyBuildMixIn$1.create(LazyBuildMixIn.java:135)
at hudson.model.RunMap.retrieve(RunMap.java:220)
at hudson.model.RunMap.retrieve(RunMap.java:57)
at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:465) - locked hudson.model.RunMap@4ca4594d
at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:448)
at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:356)
at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:332)
at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:74)
at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:63)
at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
at hudson.util.RunList.size(RunList.java:108)
at hudson.plugins.view.dashboard.stats.StatJobs$HealthStatus.getHealthStatus(StatJobs.java:54)
at hudson.plugins.view.dashboard.stats.StatJobs.getJobStat(StatJobs.java:104)
Even after the initial load, subsequent requests still take several seconds due to complete traversal of every build for every job.
"Handling GET / from 192.168.70.53 : RequestHandlerThread20 View/index.jelly Dashboard/main.jelly StatJobs/portlet.jelly StatJobs/statjobs.jelly" Id=69 Group=main RUNNABLE
at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:332)
at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:74)
at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:63)
at java.util.AbstractMap$2$1.next(AbstractMap.java:385)
at hudson.util.RunList.size(RunList.java:108)
at hudson.plugins.view.dashboard.stats.StatJobs$HealthStatus.getHealthStatus(StatJobs.java:54)
at hudson.plugins.view.dashboard.stats.StatJobs.getJobStat(StatJobs.java:104)
Code changed in jenkins
User: cburroughs
Path:
src/main/java/hudson/plugins/view/dashboard/stats/StatJobs.java
http://jenkins-ci.org/commit/dashboard-view-plugin/4eff12fc24bb09ab9c95bc0bd60eb28f3e75cd8d
Log:
optimize Job Statistics portlet
Loading every build for every job can be expensive, especially in
pathologically cases with > 10k builds per job. Fortunately the
portlet only needs to know that a non-zero number of builds exist and
not any of their details.
Problem diagnosed and fix approach proposed by Matthew Sawin
[FIXED JENKINS-26879]