Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46460

Performance issue in AsyncResourceDisposer.persist

    XMLWordPrintable

Details

    Description

      A thread dump of a Jenkins system suffering very poor performance showed many threads blocked in AsyncResourceDisposer.persist, here:

      "Computer.threadPoolForRemoting [#...]" ... waiting for monitor entry [...]
         java.lang.Thread.State: BLOCKED (on object monitor)
      	at com.thoughtworks.xstream.core.DefaultConverterLookup.lookupConverterForType(DefaultConverterLookup.java:49)
      	- waiting to lock <...> (a com.thoughtworks.xstream.core.DefaultConverterLookup)
      	at com.thoughtworks.xstream.XStream$1.lookupConverterForType(XStream.java:498)
      	at ...
      	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
      	at hudson.XmlFile.write(XmlFile.java:170)
      	at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer.persist(AsyncResourceDisposer.java:175)
      	at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer.access$300(AsyncResourceDisposer.java:65)
      	at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer$WorkItem.run(AsyncResourceDisposer.java:262)
      	at ...
      

      This is written poorly and causing massive contention on a single lock in XStream; it should be coalescing calls to persist, and preferably even delaying them by a few seconds' grace period to try to batch up as much as possible.

      The workaround is to remove this plugin. It is typically used by ws-cleanup; better to, say, use the CleanBeforeCheckout option in Git.

      Attachments

        Issue Links

          Activity

            jglick Jesse Glick created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue depends on JENKINS-19561 [ JENKINS-19561 ]
            olivergondza Oliver Gondža made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            olivergondza Oliver Gondža made changes -
            Remote Link This issue links to "resource-disposer-plugin#1 (Web Link)" [ 17662 ]
            olivergondza Oliver Gondža made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            olivergondza Oliver Gondža made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            jamesdumay James Dumay made changes -
            Remote Link This issue links to "CloudBees Internal OSS-2452 (Web Link)" [ 18313 ]

            People

              olivergondza Oliver Gondža
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: