Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info,
I got growing used metaspace while used heap is stable (after gc). see attached picture.
My java options:
-Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC
Jenkins ran out of metaspace and unresponsive finally:
2022-07-15 16:36:52.921 [Pipeline] End of Pipeline
2022-07-15 16:36:52.967 java.lang.OutOfMemoryError: Metaspace
2022-07-15 16:36:52.973 Finished: FAILURE
top class_stats after 10 hours. the used metaspace increased around 22%, see attached picture.
metaspace issue disappeared on 7/19 suddenly.
- relates to
-
JENKINS-63766 Metaspace memory leak in Pipeline when running on Java 11 (but not 8 or 17)
-
- Closed
-
loblab could you review the comments and discussion in
JENKINS-63766to see if the condition you're seeing is related to a metaspace memory leak bug in JDK 11? A workaround was implemented in Jenkins 2.346.1 and I believe that a fix is expected in Java 11.0.16