Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-75675

Refactor class loading logic in order to reduce memory consumption

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • core

      I found that Jenkins plugin classloaders consume a significant, but unnecessary, amount of memory due to storing class loading locks.

      The plugin classloader implementations (jenkins.util.URLClassLoader2 and hudson.PluginFirstClassLoader2) are based on the standard java.net.URLClassLoader, which extends the base class loading logic of java.lang.ClassLoader.

      Here is a simplified pseudocode of java.lang.ClassLoader:

      public abstract class ClassLoader {
        private final ClassLoader parent;
      
        private final ConcurrentHashMap<String, Object> parallelLockMap; 
      
      
        protected Object getClassLoadingLock(String className) {
          Object newLock = new Object();
          lock = parallelLockMap.putIfAbsent(className, newLock);
          if (lock == null) {
            lock = newLock;
          }
         
          return lock;
        }
      
        protected Class<?> loadClass(String name, boolean resolve) {
          synchronized (getClassLoadingLock(name)) {
            Class<?> c = findLoadedClass(name);
            if (c == null) {
              if (parent != null) {
                c = parent.loadClass(name, false);
              } else {
                c = findBootstrapClassOrNull(name);
              }
            } 
      
            if (c == null) {
              // If still not found, then invoke findClass in order
              // to find the class.
              c = findClass(name);
            }
            return c;
          }
      }

      As you can see, parallelLockMap retains entries for every class name ever attempted to be loaded, even if:

      • the class is located in a parent classloader,
      • the class name is invalid or nonexistent.

      On my test Jenkins instance, I have about 200 plugins, and each of them uses around 5 MB for storing parallelLockMap, resulting in ~1 GB of wasted memory.

      It appears that the standard JDK classloader implementations are well-suited for small, static class hierarchies, but not for dynamically evolving systems like Jenkins.

      I suggest overriding the getClassLoadingLock() method in jenkins.util.URLClassLoader2 to use weak references for the lock objects. This would allow unused lock entries to be garbage-collected and reduce unnecessary memory retention.

            dukhlov Dmytro
            dukhlov Dmytro
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: