Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46514

ClassLoader leak in remoting with job-dsl plugin

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Component/s: job-dsl-plugin, remoting
    • Labels:
      None
    • Environment:
      Jenkins 2.67
      job-dsl 1.64
    • Similar Issues:

      Description

      We have the same issue as JENKINS-30832 and we suspected a classloader leak, that is why we did some heap dumps. What we found is that after running the seed job (which processes N groovy scripts) M times, there were N*M GroovyClassLoaders retained. YourKit Java Profiler showed us that each of them loaded the classes (mostly closures) from one of our N groovy scripts.

      We also traced what is retaining the classloaders and we ended up at the unexportLog field of hudson.remoting.ExportTable. We didn't fully understand why was it retaining the classloaders, see the screenshot attached (the majority of LinkedList nodes was cut out). ul_rsp is one of the groovy scripts processed, the adress 172.17.0.1 corresponds to the node the seed job was running on.

      As a workaround hudson.remoting.ExportTable.unexportLogSize was set to 0 to prevent adding entries to unexportLog. Checking the heap dump revealed that the same amount of GroovyClassLoaders are present but held via weak/soft references only, making them eligible for garbage collection.

      I should also mention that the classloaders were leaked even after we got rid of the mixins in the seed job.

        Attachments

          Activity

          Hide
          daspilker Daniel Spilker added a comment -

          Hm, the dump does not contain any Groovy or Job DSL classes. I'm not sure how to debug this.

          Show
          daspilker Daniel Spilker added a comment - Hm, the dump does not contain any Groovy or Job DSL classes. I'm not sure how to debug this.
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          If this is a real case, I am pretty sure it will break with JEP-200 on Jenkins 2.102+
          Petres Andras have you already upgraded just in case?

          Show
          oleg_nenashev Oleg Nenashev added a comment - If this is a real case, I am pretty sure it will break with JEP-200 on Jenkins 2.102+ Petres Andras have you already upgraded just in case?
          Hide
          apetres Petres Andras added a comment -

          I upgraded to 2.106 and did some dumps, no ExportTable or remoting-related stuff is retaining the classloaders anymore.

          Show
          apetres Petres Andras added a comment - I upgraded to 2.106 and did some dumps, no ExportTable or remoting-related stuff is retaining the classloaders anymore.
          Hide
          daspilker Daniel Spilker added a comment -

          Petres Andras Can we close the issue?

          Show
          daspilker Daniel Spilker added a comment - Petres Andras Can we close the issue?
          Hide
          apetres Petres Andras added a comment -

          Sure!

          Show
          apetres Petres Andras added a comment - Sure!

            People

            Assignee:
            daspilker Daniel Spilker
            Reporter:
            apetres Petres Andras
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: