-
Bug
-
Resolution: Fixed
-
Minor
A Slaves nodeProperties are owned by Jenkins.
So whenever a NodeProperty is changed this causes persistance of the entire jenkins configuration rather than just Nodes which is what actually persists the NodeProperties.
Whilst this is just inefficient - if anyone wants to add a node property to new nodes created (e.g. by implementing a cloud listener) then is it possible to get a deadlock if someone also saves the Jenkins configuration at the same time. (as Jenkins.save() will lock Jenkins and then the queue, but callback for Nodes can happen with the queue locked).
"jenkins.util.Timer [#3]" id=34 (0x22) state=BLOCKED cpu=91% - waiting to lock <0x107c53da> (a hudson.model.Hudson) owned by Handling POST /configSubmit : http-8080-5 id=42 at jenkins.model.Jenkins.save(Jenkins.java:2734) at hudson.util.PersistedList.onModified(PersistedList.java:173) at hudson.util.PersistedList._onModified(PersistedList.java:181) at hudson.util.PersistedList.add(PersistedList.java:72) at com.cloudbees.librato.CloudListener.onComplete(CloudListener.java:83) at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:223) at hudson.model.Queue._withLock(Queue.java:1235) at hudson.model.Queue.withLock(Queue.java:1171) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:208) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:57) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:777) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Locked synchronizers: count = 3 - java.util.concurrent.locks.ReentrantLock$NonfairSync@2e7d6057 - java.util.concurrent.ThreadPoolExecutor$Worker@6d40c3f9 - java.util.concurrent.locks.ReentrantLock$NonfairSync@640c882b "Handling POST /configSubmit : http-8080-5" id=42 (0x2a) state=WAITING cpu=96% - waiting on <0x2e7d6057> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) - locked <0x2e7d6057> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) owned by jenkins.util.Timer [#3] id=34 at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290) at hudson.model.Queue._withLock(Queue.java:1233) at hudson.model.Queue.withLock(Queue.java:1171) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:168) at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.doConfigSubmit(Jenkins.java:2858) ...
- is related to
-
JENKINS-30060 Listeners are called back whilst holding locks which is prone to deadlocks
-
- Open
-
- links to
Code changed in jenkins
User: James Nord
Path:
core/src/main/java/hudson/model/Computer.java
core/src/main/java/hudson/model/Slave.java
core/src/main/java/jenkins/model/Jenkins.java
http://jenkins-ci.org/commit/jenkins/06f690dbb9f491d592d42f5b911ae9d49f6e1290
Log:
JENKINS-30057NodeProperties should be owned by the corresponding Saveable.Node objects are owned by the Nodes class not by Jenkins so NodeProperties
and anything modifiying them needs to be updated. Also modifying Nodes
requires a lock on the Queue so use this lock not Jenkins to prevent the
deadlock observed in
JENKINS-30057