Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32629

Deadlock between h.m.queue.FutureImpl#cancel and h.m.Queue#cancel

XMLWordPrintable

      Found from the thread dump I took when our Jenkins instance (1.609.3) didn't respond anymore. Probably a rare scenario when the stars align and the same queue item is cancelled from two different places.

      Found one Java-level deadlock:
      =============================
      "Handling POST /jenkins/queue/cancelItem from 192.168.23.74 : RequestHandlerThread[#1775]":
        waiting to lock monitor 0x00007fd2fb61f838 (object 0x000000070b2b2a28, a hudson.model.queue.FutureImpl),
        which is held by "Executor #-1 for node : executing some-job-flow #97"
      "Executor #-1 for node : executing some-job-flow #97":
        waiting for ownable synchronizer 0x00000006c03be958, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
        which is held by "Handling POST /jenkins/queue/cancelItem from 192.168.23.74 : RequestHandlerThread[#1775]"
      

      The stack traces are as follows:

      "Executor #-1 for node : executing some-job-flow #97":
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006c03be958> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
      	at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
      	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290)
      	at hudson.model.Queue.cancel(Queue.java:725)
      	at hudson.model.queue.FutureImpl.cancel(FutureImpl.java:82)
      	- locked <0x000000070b2b2a28> (a hudson.model.queue.FutureImpl)
      	- locked <0x00000006c03bdd38> (a hudson.model.Queue)
      	at java_util_concurrent_Future$cancel.call(Unknown Source)
      	at com.cloudbees.plugins.flow.JobInvocation.abort(JobInvocation.groovy:113)
      	at com.cloudbees.plugins.flow.JobInvocation$abort.call(Unknown Source)
      	at com.cloudbees.plugins.flow.FlowDSL$_killRunningJobs_closure3.doCall(FlowDSL.groovy:111)
      
      "Handling POST /jenkins/queue/cancelItem from 192.168.23.74 : RequestHandlerThread[#1775]":
      	at hudson.model.queue.FutureImpl.setAsCancelled(FutureImpl.java:89)
      	- waiting to lock <0x000000070b2b2a28> (a hudson.model.queue.FutureImpl)
      	at hudson.model.Queue$Item.cancel(Queue.java:2111)
      	at hudson.model.Queue.cancel(Queue.java:748)
      	at hudson.model.Queue.doCancelItem(Queue.java:761)
      

      Version 0.12 of the build-flow-plugin is installed. Let me know if you need additional information.

            Unassigned Unassigned
            tonis_pool Tõnis Pool
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: