-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
OS: OS Linux, 3.10.0-1160.41.1.el7.x86_64 , amd64/64 (4 cores)
Java: OpenJDK Runtime Environment, 11.0.13+8-LTS
JVM: OpenJDK 64-Bit Server VM, 11.0.13+8-LTS, mixed mode
PID of process: 997
Nb of opened files 1,402 / 8,192 ++++++++++++
Server: Server Apache Tomcat/9.0.37
Webapp context:
Start: 11/21/21, 10:17 AM
JVM arguments: --add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
-Djava.util.logging.config.file=/apps/instances/jenkins/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-Xloggc:/apps/run/jenkins_gc.log
-XX:+PrintGCDetails
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/apps/logs/jenkins/heap_dump_%p.hprof
-XX:ErrorFile=/apps/logs/jenkins/hs_err_%p.log
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-Djenkins.install.runSetupWizard=false
-Xmx8072m
-XX:MaxPermSize=1024M
-DJENKINS_HOME=/apps/jenkins/latest
-Dhudson.plugins.active_directory.ActiveDirectorySecurityRealm.forceLdaps=true
-Djava.io.tmpdir=/apps/temp
-Djava.awt.headless=true
-Dhudson.model.DirectoryBrowserSupport.CSP=
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
-Djavax.net.ssl.keyStore=/apps/conf/pki/keystore.jks
-Djavax.net.ssl.keyStorePassword=EnggApps
-Djavax.net.ssl.trustStore=/apps/conf/pki/truststore.jks
-Djavax.net.ssl.trustStorePassword=EnggApps
-Djava.security.auth.login.config=/apps/conf/java/jaas.conf
-Dignore.endorsed.dirs=
-Dcatalina.base=/apps/instances/jenkins
-Dcatalina.home=/apps/tomcat/latest
-Djava.io.tmpdir=/apps/instances/jenkins/temp
Mean age of http sessions (min): 91
Tomcat "https-jsse-nio-8443": Busy threads = 1 / 150 ++++++++++++
Bytes received = 1,732,929
Bytes sent = 560,524,404
Request count = 5,237
Error count = 92
Sum of processing times (ms) = 715,754
Max processing time (ms) = 195,250
Memory: Non heap memory = 372 Mb (Perm Gen, Code Cache),
Buffered memory = 0 Mb,
Loaded classes = 38,043,
Garbage collection time = 31,622 ms,
Process cpu time = 2,682,600 ms,
Committed virtual memory = 12,512 Mb,
Free physical memory = 2,938 Mb,
Total physical memory = 15,885 Mb
Free disk space: 267,087 Mb
Usable disk space: 251,080 MbOS: OS Linux, 3.10.0-1160.41.1.el7.x86_64 , amd64/64 (4 cores) Java: OpenJDK Runtime Environment, 11.0.13+8-LTS JVM: OpenJDK 64-Bit Server VM, 11.0.13+8-LTS, mixed mode PID of process: 997 Nb of opened files 1,402 / 8,192 ++++++++++++ Server: Server Apache Tomcat/9.0.37 Webapp context: Start: 11/21/21, 10:17 AM JVM arguments: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED -Djava.util.logging.config.file=/apps/instances/jenkins/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xloggc:/apps/run/jenkins_gc.log -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/apps/logs/jenkins/heap_dump_%p.hprof -XX:ErrorFile=/apps/logs/jenkins/hs_err_%p.log -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Djenkins.install.runSetupWizard=false -Xmx8072m -XX:MaxPermSize=1024M -DJENKINS_HOME=/apps/jenkins/latest -Dhudson.plugins.active_directory.ActiveDirectorySecurityRealm.forceLdaps=true -Djava.io.tmpdir=/apps/temp -Djava.awt.headless=true -Dhudson.model.DirectoryBrowserSupport.CSP= -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Djavax.net.ssl.keyStore=/apps/conf/pki/keystore.jks -Djavax.net.ssl.keyStorePassword=EnggApps -Djavax.net.ssl.trustStore=/apps/conf/pki/truststore.jks -Djavax.net.ssl.trustStorePassword=EnggApps -Djava.security.auth.login.config=/apps/conf/java/jaas.conf -Dignore.endorsed.dirs= -Dcatalina.base=/apps/instances/jenkins -Dcatalina.home=/apps/tomcat/latest -Djava.io.tmpdir=/apps/instances/jenkins/temp Mean age of http sessions (min): 91 Tomcat "https-jsse-nio-8443": Busy threads = 1 / 150 ++++++++++++ Bytes received = 1,732,929 Bytes sent = 560,524,404 Request count = 5,237 Error count = 92 Sum of processing times (ms) = 715,754 Max processing time (ms) = 195,250 Memory: Non heap memory = 372 Mb (Perm Gen, Code Cache), Buffered memory = 0 Mb, Loaded classes = 38,043, Garbage collection time = 31,622 ms, Process cpu time = 2,682,600 ms, Committed virtual memory = 12,512 Mb, Free physical memory = 2,938 Mb, Total physical memory = 15,885 Mb Free disk space: 267,087 Mb Usable disk space: 251,080 Mb
I noticed that the heap memory is growing, with a lot of Step's like objects
org.jenkinsci.plugins.workflow.cps.nodes.Step*
Even if the jenkins is idle (no activity for a while) those objects remain in memory.
After a while it uses all the heap and push it up to its max, slowing down then the jenkins primary (UI is getting slower and slower).
So I have created a heap dump and analyzed it with Eclise Memory Analyzer, which reported those objects as the most used one.
Is there any configuration or JVM parameter that I would have missed to improve the performances ? (see environment information for details on the current environment)
According to the info this is the version installed:
Implementation-Version: 2633.v6baeedc13805
Is there a mechanism / code that is supposed to cleanup those objects once the build completed ? where is that code located ? Why it is still in memory otherwise ?
