Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63975

Jenkins controller failing with java.lang.OutOfMemoryError: GC overhead limit exceeded

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • Jenkins 2.235.3, Using Docker container

      Hi,

      Since last few weeks we are facing issue with Jenkins master, it crashes every now and then with exception

      java.lang.OutOfMemoryError: GC overhead limit exceeded

      Below is what we found in logs, attached file "log_trace.txt" contains full exception stack trace.

      2020-10-14 15:02:48.604+0000 [id=13455] WARNING h.i.i.InstallUncaughtExceptionHandler#handleException org.apache.commons.jelly.JellyTagException: jar:file:/var/jenkins_home/war/WEB-INF/lib/jenkins-core-2.235.3.jar!/hudson/model/View/index.jelly:42:43: <st:include> org.apache.commons.jelly.JellyTagException: jar:file:/var/jenkins_home/war/WEB-INF/lib/jenkins-core-2.235.3.jar!/lib/hudson/projectViewRow.jelly:35:52: <st:include> java.lang.OutOfMemoryError: GC overhead limit exceeded     at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:726)     at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:281)     at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)     at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
      
      

      When created heap dump and histogram below stack trace is looks suspicious with large size. (attached the full histogram details)

      3: 1007784 32249088 java.lang.StackTraceElement

      Also attached the GC Root from dump file for same stack trace. We have observed same trace in all our slave dumpExportTable.

      Memory parameters configured for Jenkins "-Xmx4096m -XX:MaxPermSize=1024m"

      Is this a memory leak causing the failures ? During the same time our CPU usage also clocks as 100%, are they both related to each other ?

      Please note this Jenkins instance is running on a dedicated VM.

       

        1. node_dump_export_table.txt
          1.05 MB
        2. log_trace.txt
          48 kB
        3. heap_histo.txt
          1.31 MB
        4. heap_dump_gc_root.PNG
          heap_dump_gc_root.PNG
          171 kB
        5. gc.log
          5.04 MB

            Unassigned Unassigned
            rgaduput Reddysekhar Gaduputi
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: