Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30060

Listeners are called back whilst holding locks which is prone to deadlocks

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • core

      many listeners are called whilst locks are held which leads to deadlocks (see JENKINS-30057 for one example).

      A listener may reasonably want to perform some action based on this which inevitably requires some call back to Jenkins which will require a different lock. However the listener is not at this point in control of what locks are held when it is called so it needs to jump through hoops in order to do things correctly. Whilst looking at listeners I have found zero instances of Listeners that do jump through the hoops - and it would seem like Jenkins should actually be handling this on behalf of the listener.

      For example Nodes may be provisioned with the Queue lock held - and if some jenkins configuration needs to be modified will call locking order of Queue -> Jenkins. Whilst someone saving Jenkins Clouds could lock Jenkins -> Queue.

      Whilst we can chase our tails unpicking these deadlocks individually I have seen 3 of these recently and it would be much better to ensure that the callbacks happened without any locks held.

          [JENKINS-30060] Listeners are called back whilst holding locks which is prone to deadlocks

          Jesse Glick added a comment -

          Agreed. While it is generally a poor idea for a listener implementation to do anything that might acquire a lock, in practice this is hard to do; and using Timer.get to run the body asynchronously just makes for serious headaches in functional tests. The best option, when possible, is to call listeners with no locks held.

          Jesse Glick added a comment - Agreed. While it is generally a poor idea for a listener implementation to do anything that might acquire a lock, in practice this is hard to do; and using Timer.get to run the body asynchronously just makes for serious headaches in functional tests. The best option, when possible, is to call listeners with no locks held.

            Unassigned Unassigned
            teilo James Nord
            Votes:
            2 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: