-
Bug
-
Resolution: Fixed
-
Major
-
None
-
-
support-core-2.60
OOM situation investigation revealed that com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection:
println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size()
Values from some of the instances with varying size/uptime:
Listeneres 1990 Computers 5 --- Listeneres 62 Computers 33 --- Listeneres 16521 Computers 16
The instance torn down had 30K instances.
[JENKINS-58528] Channel listeners (SlaveCommandStatistics.Statistics) retained after node removal
| Description |
Original:
OOM situation investigation revealed that \{com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics}} is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection: {noformat} println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size() {noformat} Values from some of the instances with varying size/uptime: {noformat} Listeneres 1990 Computers 5 --- println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size() --- println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size() {noformat} The instance torn down had 30K instances. |
New:
OOM situation investigation revealed that \{com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics}} is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection: {noformat} println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size() {noformat} Values from some of the instances with varying size/uptime: {noformat} Listeneres 1990 Computers 5 --- Listeneres 62 Computers 33 --- Listeneres 16521 Computers 16 {noformat} The instance torn down had 30K instances. |
| Description |
Original:
OOM situation investigation revealed that \{com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics}} is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection: {noformat} println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size() {noformat} Values from some of the instances with varying size/uptime: {noformat} Listeneres 1990 Computers 5 --- Listeneres 62 Computers 33 --- Listeneres 16521 Computers 16 {noformat} The instance torn down had 30K instances. |
New:
OOM situation investigation revealed that {{com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics}} is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection: {noformat} println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size() println "Computers " + Jenkins.instance.computers.size() {noformat} Values from some of the instances with varying size/uptime: {noformat} Listeneres 1990 Computers 5 --- Listeneres 62 Computers 33 --- Listeneres 16521 Computers 16 {noformat} The instance torn down had 30K instances. |
| Status | Original: Open [ 1 ] | New: In Progress [ 3 ] |
Yes we need to preserve statistics after agent removal. I commented in more detail in the PR.
| Released As | New: support-core-2.60 | |
| Resolution | New: Fixed [ 1 ] | |
| Status | Original: In Progress [ 3 ] | New: Resolved [ 5 ] |
| Status | Original: Resolved [ 5 ] | New: Closed [ 6 ] |
jglick, as an author of the report, would you say this makes sense preserving after computer is gone (vs. only while it is present)? I am considering when to rotate the records - at node removal or keeping it around for a while to ease investigation...