Jenkins controller failing with java.lang.OutOfMemoryError: GC overhead limit exceeded

This issue is archived. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      Hi,

      Since last few weeks we are facing issue with Jenkins master, it crashes every now and then with exception

      java.lang.OutOfMemoryError: GC overhead limit exceeded

      Below is what we found in logs, attached file "log_trace.txt" contains full exception stack trace.

      2020-10-14 15:02:48.604+0000 [id=13455] WARNING h.i.i.InstallUncaughtExceptionHandler#handleException org.apache.commons.jelly.JellyTagException: jar:file:/var/jenkins_home/war/WEB-INF/lib/jenkins-core-2.235.3.jar!/hudson/model/View/index.jelly:42:43: <st:include> org.apache.commons.jelly.JellyTagException: jar:file:/var/jenkins_home/war/WEB-INF/lib/jenkins-core-2.235.3.jar!/lib/hudson/projectViewRow.jelly:35:52: <st:include> java.lang.OutOfMemoryError: GC overhead limit exceeded     at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:726)     at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:281)     at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)     at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
      
      

      When created heap dump and histogram below stack trace is looks suspicious with large size. (attached the full histogram details)

      3: 1007784 32249088 java.lang.StackTraceElement

      Also attached the GC Root from dump file for same stack trace. We have observed same trace in all our slave dumpExportTable.

      Memory parameters configured for Jenkins "-Xmx4096m -XX:MaxPermSize=1024m"

      Is this a memory leak causing the failures ? During the same time our CPU usage also clocks as 100%, are they both related to each other ?

      Please note this Jenkins instance is running on a dedicated VM.

       

        1. gc.log
          5.04 MB
        2. heap_dump_gc_root.PNG
          heap_dump_gc_root.PNG
          171 kB
        3. heap_histo.txt
          1.31 MB
        4. log_trace.txt
          48 kB
        5. node_dump_export_table.txt
          1.05 MB

            Assignee:
            Unassigned
            Reporter:
            Reddysekhar Gaduputi
            Archiver:
            Jenkins Service Account

              Created:
              Updated:
              Archived: