Critical Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info,
I got growing used metaspace while used heap is stable (after gc). see attached picture.
My java options:
-Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC
Jenkins ran out of metaspace and unresponsive finally:
2022-07-15 16:36:52.921 [Pipeline] End of Pipeline
2022-07-15 16:36:52.967 java.lang.OutOfMemoryError: Metaspace
2022-07-15 16:36:52.973 Finished: FAILURE
top class_stats after 10 hours. the used metaspace increased around 22%, see attached picture.
metaspace issue disappeared on 7/19 suddenly.
- relates to
-
JENKINS-63766 Metaspace memory leak in Pipeline when running on Java 11 (but not 8 or 17)
-
- Closed
-
[JENKINS-69022] Metaspace leak on Jenkins 2.346.1/Java11
| Description | Original: Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with | New: Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with GC.heap_info |
| Description | Original: Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with GC.heap_info |
New:
Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. |
| Attachment | New: metaspace.png [ 58486 ] |
| Description |
Original:
Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. |
New:
!metaspace.png! Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. see attached picture. |
| Description |
Original:
!metaspace.png! Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. see attached picture. |
New:
!metaspace.png! Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. see attached picture. My java options: -Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC Jenkins will run out of metaspace and stop response finally. |
| Description |
Original:
!metaspace.png! Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. see attached picture. My java options: -Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC Jenkins will run out of metaspace and stop response finally. |
New:
Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. see attached picture. My java options: -Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC Jenkins will run out of metaspace and stop response finally. |
| Description |
Original:
Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is plain. see attached picture. My java options: -Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC Jenkins will run out of metaspace and stop response finally. |
New:
Running Jenkins with hundreds of nodes and jobs, monitoring metaspace and heap with
jcmd GC.heap_info, I got growing used metaspace while used heap is stable (after gc). see attached picture. My java options: -Xmx400g -Xms200g -XX:MetaspaceSize=8192m -XX:MaxMetaspaceSize=8192m -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC Jenkins ran out of metaspace and unresponsive finally: 2022-07-15 16:36:52.921 [Pipeline] End of Pipeline 2022-07-15 16:36:52.967 java.lang.OutOfMemoryError: Metaspace 2022-07-15 16:36:52.973 Finished: FAILURE |
| Labels | New: 2.346.1 leak memory metaspace |
| Link |
New:
This issue relates to |
| Attachment | New: class_stats.png [ 58502 ] |