-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Ubuntu 12.04, Oracle Java 7
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
When a slave is running a build in a multijob plugin, the monitoring thread on the master eats up 100% of a CPU core.
This scales linearly-- so when I have 4 parallel jobs running, jenkins master is using 400% cpu.
Attaching screen cap of profiler output, as well as YourKit profiler snapshot.
[JENKINS-21649] High CPU Usage on master while slaves working (multijob)
Seeing the same on Jenkins 1.532.1 with MultiJob PLugin 1.12.
Typically its the pool threads that are heavy on CPU...
"pool-26-thread-1" prio=5 RUNNABLE java.util.Hashtable.get(Unknown Source) java.text.NumberFormat.getInstance(Unknown Source) java.text.NumberFormat.getInstance(Unknown Source) java.text.MessageFormat.subformat(Unknown Source) java.text.MessageFormat.format(Unknown Source) java.text.Format.format(Unknown Source) java.text.MessageFormat.format(Unknown Source) org.jvnet.localizer.ResourceBundleHolder.format(ResourceBundleHolder.java:139) hudson.Messages.Util_second(Messages.java:405) hudson.Util.getTimeSpanString(Util.java:682) hudson.model.Run.getDurationString(Run.java:697) com.tikal.jenkins.plugins.multijob.MultiJobBuilder.updateSubBuild(MultiJobBuilder.java:292) com.tikal.jenkins.plugins.multijob.MultiJobBuilder.access$000(MultiJobBuilder.java:60) com.tikal.jenkins.plugins.multijob.MultiJobBuilder$SubJobWorker.run(MultiJobBuilder.java:200) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) java.lang.Thread.run(Unknown Source) "pool-26-thread-2" prio=5 WAITING hudson.remoting.AsyncFutureImpl.isCancelled(AsyncFutureImpl.java:64) com.tikal.jenkins.plugins.multijob.MultiJobBuilder$SubJobWorker.run(MultiJobBuilder.java:194) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) java.lang.Thread.run(Unknown Source)
Dan Dumont
added a comment - There's more information in JENKINS-21798
The thread trace is misleading, as the threads round-robin the work.
Dan Dumont
added a comment - There's more information in JENKINS-21798
The thread trace is misleading, as the threads round-robin the work. 
hagzag
added a comment - Itai/Alex could you please look into this, looks like a show stopper to me (if we ca re-produce).
Tx
hagzag
added a comment - Itai/Alex could you please look into this, looks like a show stopper to me (if we ca re-produce).
Tx Seem to be a fix here: https://github.com/jenkinsci/tikal-multijob-plugin/pull/49
Is that good? Any ETA on this fix, 1.13?
Code changed in jenkins
User: Nicolas Morey-Chaisemartin
Path:
src/main/java/com/tikal/jenkins/plugins/multijob/MultiJobBuilder.java
http://jenkins-ci.org/commit/tikal-multijob-plugin/f9a88fd8a6b094b4e817bcc4a0fc19ab91be2ddb
Log:
MultiJobBuilder.java: Sleep in the subtask polling loop
Without the sleep, the thread keeps polling and updating the subtask which causes a very high CPU usage.
This fixes JENKINS-21649 and JENKINS-21798
Signed-off-by: Nicolas Morey-Chaisemartin <nmorey@kalray.eu>
I have made the same observation using:
Jenkins: 1.549
Multijob Plugin: 1.11