Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46514

ClassLoader leak in remoting with job-dsl plugin

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Cannot Reproduce
    • job-dsl-plugin, remoting
    • None
    • Jenkins 2.67
      job-dsl 1.64

    Description

      We have the same issue as JENKINS-30832 and we suspected a classloader leak, that is why we did some heap dumps. What we found is that after running the seed job (which processes N groovy scripts) M times, there were N*M GroovyClassLoaders retained. YourKit Java Profiler showed us that each of them loaded the classes (mostly closures) from one of our N groovy scripts.

      We also traced what is retaining the classloaders and we ended up at the unexportLog field of hudson.remoting.ExportTable. We didn't fully understand why was it retaining the classloaders, see the screenshot attached (the majority of LinkedList nodes was cut out). ul_rsp is one of the groovy scripts processed, the adress 172.17.0.1 corresponds to the node the seed job was running on.

      As a workaround hudson.remoting.ExportTable.unexportLogSize was set to 0 to prevent adding entries to unexportLog. Checking the heap dump revealed that the same amount of GroovyClassLoaders are present but held via weak/soft references only, making them eligible for garbage collection.

      I should also mention that the classloaders were leaked even after we got rid of the mixins in the seed job.

      Attachments

        Activity

          Hm, the dump does not contain any Groovy or Job DSL classes. I'm not sure how to debug this.

          daspilker Daniel Spilker added a comment - Hm, the dump does not contain any Groovy or Job DSL classes. I'm not sure how to debug this.
          oleg_nenashev Oleg Nenashev added a comment -

          If this is a real case, I am pretty sure it will break with JEP-200 on Jenkins 2.102+
          apetres have you already upgraded just in case?

          oleg_nenashev Oleg Nenashev added a comment - If this is a real case, I am pretty sure it will break with JEP-200 on Jenkins 2.102+ apetres have you already upgraded just in case?
          apetres Petres Andras added a comment -

          I upgraded to 2.106 and did some dumps, no ExportTable or remoting-related stuff is retaining the classloaders anymore.

          apetres Petres Andras added a comment - I upgraded to 2.106 and did some dumps, no ExportTable or remoting-related stuff is retaining the classloaders anymore.

          apetres Can we close the issue?

          daspilker Daniel Spilker added a comment - apetres Can we close the issue?
          apetres Petres Andras added a comment -

          Sure!

          apetres Petres Andras added a comment - Sure!

          People

            daspilker Daniel Spilker
            apetres Petres Andras
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: