Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28844

The curious case of the Channel memory cycles



    • Similar Issues:


      • In larger installations of Jenkins the complex cycles of object references containing classloaders containing references to the channel which contains references back to the classloaders can confuse the garbage collector.
      • The confusion arrises because classloaders are considered live while there are objects from the classloader live and the RemoteClassLoader and RemoteInvocationHandler's proxy both need a reference to the Channel which typically can be storing either some of the objects from the RemoteClassLoader or the RemoteInvocationHandler's proxy instances in the Channel properties. Thus although the entire subgraph is no longer live, the cycle is not detected as ClassLoaders are treated separately. The issue is worse in Java 6/7 where the ClassLoaders are on the PermGen heap though it is also present in Java 8.
      • Typically multiple out of memory errors can cause the cycle to ultimately get cleaned up as one half gets cleared out breaking the cycle enough so that a subsequent out of memory removes the second half leaving a third out of memory to clear out the ClassLoaders... but after three out of memory errors in a row, Jenkins itself is typically well dead.
      • I suspect this issue is related to the eventual OOM that JNLP slaves die from after many reconnects

      CloudBees Ref: ZD-26018


          Issue Links


            stephenconnolly Stephen Connolly created issue -
            stephenconnolly Stephen Connolly made changes -
            Field Original Value New Value
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Labels memory-leak
            stephenconnolly Stephen Connolly made changes -
            Labels memory-leak lts-candidate memory-leak
            scm_issue_link SCM/JIRA link daemon made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            olivergondza Oliver Gond┼ża made changes -
            Labels lts-candidate memory-leak 1.609.3-fixed memory-leak
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-28429 [ JENKINS-28429 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-26855 [ JENKINS-26855 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-26855 [ JENKINS-26855 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 163704 ] JNJira + In-Review [ 197284 ]
            stephenconnolly Stephen Connolly made changes -
            Status Resolved [ 5 ] Closed [ 6 ]


              stephenconnolly Stephen Connolly
              stephenconnolly Stephen Connolly
              0 Vote for this issue
              5 Start watching this issue