A thread dump of a Jenkins system suffering very poor performance showed many threads blocked in AsyncResourceDisposer.persist, here:
"Computer.threadPoolForRemoting [#...]" ... waiting for monitor entry [...]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.thoughtworks.xstream.core.DefaultConverterLookup.lookupConverterForType(DefaultConverterLookup.java:49)
- waiting to lock <...> (a com.thoughtworks.xstream.core.DefaultConverterLookup)
at com.thoughtworks.xstream.XStream$1.lookupConverterForType(XStream.java:498)
at ...
at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
at hudson.XmlFile.write(XmlFile.java:170)
at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer.persist(AsyncResourceDisposer.java:175)
at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer.access$300(AsyncResourceDisposer.java:65)
at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer$WorkItem.run(AsyncResourceDisposer.java:262)
at ...
This is written poorly and causing massive contention on a single lock in XStream; it should be coalescing calls to persist, and preferably even delaying them by a few seconds' grace period to try to batch up as much as possible.
The workaround is to remove this plugin. It is typically used by ws-cleanup; better to, say, use the CleanBeforeCheckout option in Git.
Bug
Critical
JENKINS-19561 Unsafe & inefficient concurrency in XStream
