Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58528

Channel listeners (SlaveCommandStatistics.Statistics) retained after node removal

    XMLWordPrintable

Details

    • support-core-2.60

    Description

      OOM situation investigation revealed that com.cloudbees.jenkins.support.impl.SlaveCommandStatistics#statistics is always growing. For instances with dynamically provisioned nodes this is effectively a memory leak. Detection:

      println "Listeneres " + ExtensionList.lookupSingleton(com.cloudbees.jenkins.support.impl.SlaveCommandStatistics.class).@statistics.size()
      println "Computers " + Jenkins.instance.computers.size()
      

      Values from some of the instances with varying size/uptime:

      Listeneres 1990
      Computers 5
      ---
      Listeneres 62
      Computers 33
      ---
      Listeneres 16521
      Computers 16
      

       

      The instance torn down had 30K instances.

      Attachments

        Activity

          jglick, as an author of the report, would you say this makes sense preserving after computer is gone (vs. only while it is present)? I am considering when to rotate the records - at node removal or keeping it around for a while to ease investigation...

          olivergondza Oliver Gondža added a comment - jglick , as an author of the report, would you say this makes sense preserving after computer is gone (vs. only while it is present)? I am considering when to rotate the records - at node removal or keeping it around for a while to ease investigation...
          olivergondza Oliver Gondža added a comment - Fix proposed: https://github.com/jenkinsci/support-core-plugin/pull/180
          jglick Jesse Glick added a comment -

          Yes we need to preserve statistics after agent removal. I commented in more detail in the PR.

          jglick Jesse Glick added a comment - Yes we need to preserve statistics after agent removal. I commented in more detail in the PR.

          People

            olivergondza Oliver Gondža
            olivergondza Oliver Gondža
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: