Details
-
Bug
-
Status: Resolved (View Workflow)
-
Critical
-
Resolution: Fixed
-
[chris@fen-vz-jenkins ~]$ java -version
java version "1.6.0_22"
OpenJDK Runtime Environment (IcedTea6 1.10.8) (rhel-1.27.1.10.8.el5_8-i386)
OpenJDK Server VM (build 20.0-b11, mixed mode)
[chris@fen-vz-jenkins ~]$ uname -a
Linux fen-vz-jenkins.fen.aptivate.org 2.6.32-7-pve #1 SMP Mon Feb 13 07:33:21 CET 2012 i686 i686 i386 GNU/Linux
[chris@fen-vz-jenkins ~]$ cat /etc/issue
CentOS release 5.8 (Final)[ chris@fen-vz-jenkins ~]$ java -version java version "1.6.0_22" OpenJDK Runtime Environment (IcedTea6 1.10.8) (rhel-1.27.1.10.8.el5_8-i386) OpenJDK Server VM (build 20.0-b11, mixed mode) [ chris@fen-vz-jenkins ~]$ uname -a Linux fen-vz-jenkins.fen.aptivate.org 2.6.32-7-pve #1 SMP Mon Feb 13 07:33:21 CET 2012 i686 i686 i386 GNU/Linux [ chris@fen-vz-jenkins ~]$ cat /etc/issue CentOS release 5.8 (Final)
Description
Jenkins starts using 100% CPU after a few days. Using jstack I see several threads trying to write compressed output, and apparently not changing over time:
- java.util.zip.Deflater.deflate(byte[], int, int) @bci=55, line=322 (Compiled frame; information may be imprecise)
- java.util.zip.DeflaterOutputStream.deflate() @bci=14, line=176 (Compiled frame)
- java.util.zip.DeflaterOutputStream.write(byte[], int, int) @bci=108, line=135 (Compiled frame)
- java.util.zip.GZIPOutputStream.write(byte[], int, int) @bci=4, line=89 (Compiled frame)
- org.kohsuke.stapler.compression.FilterServletOutputStream.write(byte[], int, int) @bci=7, line=31 (Compiled frame)
- sun.nio.cs.StreamEncoder.writeBytes() @bci=120, line=220 (Interpreted frame)
- sun.nio.cs.StreamEncoder.implClose() @bci=84, line=315 (Interpreted frame)
- sun.nio.cs.StreamEncoder.close() @bci=18, line=148 (Interpreted frame)
- java.io.OutputStreamWriter.close() @bci=4, line=233 (Interpreted frame)
- java.io.PrintWriter.close() @bci=21, line=312 (Interpreted frame)
- org.kohsuke.stapler.compression.CompressionFilter.reportException(java.lang.Exception, javax.servlet.http.HttpServletResponse) @bci=112, line=77 (Interpreted frame)
- org.kohsuke.stapler.compression.CompressionFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=57, line=53 (Compiled frame)
- winstone.FilterConfiguration.execute(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=25, line=194 (Compiled frame)
- winstone.RequestDispatcher.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=48, line=366 (Compiled frame)
- hudson.util.CharacterEncodingFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=43, line=81 (Compiled frame)
- winstone.FilterConfiguration.execute(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=25, line=194 (Compiled frame)
- winstone.RequestDispatcher.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=48, line=366 (Compiled frame)
- winstone.RequestDispatcher.forward(javax.servlet.ServletRequest, javax.servlet.ServletResponse) @bci=483, line=331 (Compiled frame)
- winstone.RequestHandlerThread.processRequest(winstone.WebAppConfiguration, winstone.WinstoneRequest, winstone.WinstoneResponse, java.lang.String) @bci=38, line=215 (Compiled frame)
- winstone.RequestHandlerThread.run() @bci=631, line=138 (Compiled frame)
- java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=471 (Interpreted frame)
- java.util.concurrent.FutureTask$Sync.innerRun() @bci=29, line=334 (Interpreted frame)
- java.util.concurrent.FutureTask.run() @bci=4, line=166 (Interpreted frame)
- winstone.BoundedExecutorService$1.run() @bci=4, line=77 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=46, line=1110 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=603 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=679 (Interpreted frame)
I'm suspecting but not 100% sure that these threads are in an infinite loop (livelocked). I'm struggling to see what other threads might be doing this.
This JVM was not started with debugging enabled to attach a debugger for analysis. I've enabled it now. Stack traces attached as files below.
Attachments
Issue Links
- depends on
-
JENKINS-19473 failed to archive slave artifacts. Unexpected end of ZLIB input stream
-
- Closed
-
- is duplicated by
-
JENKINS-14361 100% CPU load during org.kohsuke.stapler.compression.CompressionFilter.reportException
-
- Resolved
-
-
JENKINS-18513 Slowness in Jenkins
-
- Resolved
-
-
JENKINS-19346 100% on one CPU
-
- Resolved
-
- is related to
-
JENKINS-13625 ERR_CONTENT_DECODING_FAILED returned on testResults and console output after Jenkins reload
-
- Closed
-
-
JENKINS-17349 CPU on master node keeps increasing and never come down
-
- Resolved
-
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/hudson/maven/AbstractMavenProcessFactory.java
src/main/java/hudson/maven/AggregatingClassLoader.java
src/main/java/hudson/maven/Maven3Builder.java
src/main/java/hudson/maven/Maven3ProcessFactory.java
src/main/java/hudson/maven/MavenBuildInformation.java
src/main/java/hudson/maven/MavenEmbedderRequest.java
src/main/java/hudson/maven/MavenModuleSet.java
src/main/java/hudson/maven/MavenModuleSetBuild.java
src/main/java/hudson/maven/MavenProbeAction.java
src/main/java/hudson/maven/MavenProcessFactory.java
src/main/java/hudson/maven/MavenUtil.java
src/main/java/hudson/maven/PlexusModuleContributor.java
src/main/java/hudson/maven/reporters/MavenAbstractArtifactRecord.java
src/main/java/hudson/maven/reporters/MavenFingerprinter.java
src/main/java/hudson/maven/reporters/TestMojo.java
src/main/resources/hudson/maven/MavenBuild/executedMojos_fr.properties
src/main/resources/hudson/maven/MavenModuleSetBuild/main_pl.properties
src/main/resources/hudson/maven/Messages.properties
src/main/resources/hudson/maven/Messages_es.properties
src/main/resources/hudson/maven/Messages_zh_TW.properties
src/main/resources/hudson/maven/reporters/MavenAbstractArtifactRecord/index_fr.properties
src/test/java/hudson/maven/reporters/SurefireArchiverUnitTest.java
src/test/java/hudson/maven/reporters/TestMojoTest.java
http://jenkins-ci.org/commit/maven-plugin/14863a0498e0e6c389f2779b9150e9280b363d3e
Log:
Merge branch 'master' into
JENKINS-14362-jzlibOriginally-Committed-As: dedb24ef034a04fea288eaa50089e8e774714e99