Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58528

Channel listeners (SlaveCommandStatistics.Statistics) retained after node removal

    XMLWordPrintable

Details

    • support-core-2.60

    Description

      OOM situation investigation revealed that com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection:

      println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size()
      println "Computers " + Jenkins.instance.computers.size()
      

      Values from some of the instances with varying size/uptime:

      Listeneres 1990
      Computers 5
      ---
      Listeneres 62
      Computers 33
      ---
      Listeneres 16521
      Computers 16
      

       

      The instance torn down had 30K instances.

      Attachments

        Activity

          jglick Jesse Glick added a comment -

          Yes we need to preserve statistics after agent removal. I commented in more detail in the PR.

          jglick Jesse Glick added a comment - Yes we need to preserve statistics after agent removal. I commented in more detail in the PR.
          olivergondza Oliver Gondža added a comment - Fix proposed: https://github.com/jenkinsci/support-core-plugin/pull/180

          jglick, as an author of the report, would you say this makes sense preserving after computer is gone (vs. only while it is present)? I am considering when to rotate the records - at node removal or keeping it around for a while to ease investigation...

          olivergondza Oliver Gondža added a comment - jglick , as an author of the report, would you say this makes sense preserving after computer is gone (vs. only while it is present)? I am considering when to rotate the records - at node removal or keeping it around for a while to ease investigation...

          People

            olivergondza Oliver Gondža
            olivergondza Oliver Gondža
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: