Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34433

Pipeline locks not signalled on manual unreserving of resources

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • Jenkins 1.642.4.2 with Cloudbees Enterprise 15.11
      Lockable Resource plugin 1.8
      Lockable Resource plugin 2.2
    • Pipeline - April 2018
    • lockable-resources 2.3

    Description

      Add a lockable resource named "test_res"

      Add a pipeline job with only this code:

      node {

      lock('test_res') {
      echo 'locked'
      }
      echo 'unlocked'
      }

      Run this job. It should work fine.

      Now perform a manual reserve of the resource via the lockable resource interface.
      Start the job again.
      It correctly waits for the resource:
      [Pipeline] node {
      [Pipeline] Lock shared resource : Start
      Trying to acquire lock on [test_res]
      [test_res] is locked, waiting...

      Now go back to the Lockable Resource user interface.
      Manually unlock the resource.

      Expected result: the locked job should retry the lock, see that it has been freed, continue (locking the resource and freeing it) and finish correctly.

      Observed result: after waiting for > 5 minutes, nothing changed. The job stayed in exactly the same situation (output as above) and had to be cancelled.

      Attachments

        Issue Links

          Activity

            xvdessel Xavier Van Dessel created issue -
            xvdessel Xavier Van Dessel made changes -
            Field Original Value New Value
            Component/s lockable-resources-plugin [ 18222 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 170547 ] JNJira + In-Review [ 183951 ]
            rndgstn Ron Dagostino added a comment - Another example, this time with a timeout: https://groups.google.com/d/msg/jenkinsci-users/MIVk-44cUcA/snKuC3PEAwAJ
            recampbell Ryan Campbell made changes -
            Component/s pipeline-utility-steps-plugin [ 21135 ]
            recampbell Ryan Campbell made changes -
            Assignee rsandell [ rsandell ]
            amuniz Antonio Muñiz added a comment - This may be fixed by https://github.com/jenkinsci/lockable-resources-plugin/pull/46 Needs checking.
            zsoltilles Zsolt Illés added a comment -

            In my case this does not solve the manual resource unlocking problem.

            zsoltilles Zsolt Illés added a comment - In my case this does not solve the manual resource unlocking problem.
            roivanov Roman Ivanov added a comment -

            This issue still there. Jenkins 2.62, Lockable Resources plugin 2.0

            {{Trying to acquire lock on [test] }}

            Found 0 available resource(s). Waiting for correct amount: 1.

            [test] is locked, waiting...

            roivanov Roman Ivanov added a comment - This issue still there. Jenkins 2.62, Lockable Resources plugin 2.0 {{Trying to acquire lock on [test] }} Found 0 available resource(s). Waiting for correct amount: 1. [test] is locked, waiting...

            Technically, it does seem to retry the lock - provided another pipeline signals that the lock is now released.

            My theory, that I have not tested in the slightest yet, is that the plugin does not broadcast the lock release the same way a pipeline does.

            zmeggyesi Zalan Meggyesi added a comment - Technically, it does seem to retry the lock - provided another pipeline signals that the lock is now released. My theory, that I have not tested in the slightest yet, is that the plugin does not broadcast the lock release the same way a pipeline does.
            dolphy David Crouch added a comment -

            I am experiencing this issue as well. Any pipeline that becomes locked on a manually reserved resource will never unlock, even after the resource is released.This functionally makes the manual lock reserve inoperable. I believe another pipeline job locking and then unlocking the resource will "signal" the deadlocked job, but this still results in a mostly unusable manual reserve.

            I would suggest changing the title of this issue to reflect the scope and severity, something to the effect of:

            Pipeline jobs will deadlock when trying to lock manually reserved resource

             

            dolphy David Crouch added a comment - I am experiencing this issue as well. Any pipeline that becomes locked on a manually reserved resource will never unlock, even after the resource is released.This functionally makes the manual lock reserve inoperable. I believe another pipeline job locking and then unlocking the resource will "signal" the deadlocked job, but this still results in a mostly unusable manual reserve. I would suggest changing the title of this issue to reflect the scope and severity, something to the effect of: Pipeline jobs will deadlock when trying to lock manually reserved resource  
            abayer Andrew Bayer made changes -
            Assignee Andrew Bayer [ abayer ]
            abayer Andrew Bayer added a comment -

            zmeggyesi - that sounds right. I'll need to dig into this later today, but I think this shouldn't be too hard to fix.

            abayer Andrew Bayer added a comment - zmeggyesi - that sounds right. I'll need to dig into this later today, but I think this shouldn't be too hard to fix.
            abayer Andrew Bayer made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            abayer Andrew Bayer made changes -
            Summary pipeline lock does not retry Pipeline locks not signalled on manual unreserving of resources
            abayer Andrew Bayer made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            abayer Andrew Bayer added a comment - PR up at https://github.com/jenkinsci/lockable-resources-plugin/pull/70
            abayer Andrew Bayer made changes -
            Remote Link This issue links to "PR #70 (Web Link)" [ 17462 ]
            giorgiosironi Giorgio Sironi made changes -
            Link This issue is duplicated by JENKINS-46745 [ JENKINS-46745 ]
            jamesdumay James Dumay made changes -
            Sprint Pipeline - December [ 446 ]
            jamesdumay James Dumay made changes -
            Rank Ranked lower
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal CD-216 (Web Link)" [ 19050 ]
            vivek Vivek Pandey made changes -
            Sprint Pipeline - December [ 446 ] Pipeline - March 2018 [ 506 ]
            abayer Andrew Bayer added a comment -

            Gonna take some time this week to resurrect this.

            abayer Andrew Bayer added a comment - Gonna take some time this week to resurrect this.
            abayer Andrew Bayer added a comment -

            And PR updated.

            abayer Andrew Bayer added a comment - And PR updated.

            Code changed in jenkins
            User: Andrew Bayer
            Path:
            src/main/java/org/jenkins/plugins/lockableresources/LockableResourcesManager.java
            src/test/java/org/jenkins/plugins/lockableresources/LockStepTest.java
            http://jenkins-ci.org/commit/lockable-resources-plugin/d18f7baebe46220404a10175303bb0de41cdddb5
            Log:
            [FIXED JENKINS-34433] Signal queued Pipeline tasks on unreserve

            Previously, we had already been signalling queued Pipeline tasks on
            unlocking, but not on unreserving. So...let's do that.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: src/main/java/org/jenkins/plugins/lockableresources/LockableResourcesManager.java src/test/java/org/jenkins/plugins/lockableresources/LockStepTest.java http://jenkins-ci.org/commit/lockable-resources-plugin/d18f7baebe46220404a10175303bb0de41cdddb5 Log: [FIXED JENKINS-34433] Signal queued Pipeline tasks on unreserve Previously, we had already been signalling queued Pipeline tasks on unlocking, but not on unreserving. So...let's do that.
            abayer Andrew Bayer added a comment -

            Will be in next release (2.3?)

            abayer Andrew Bayer added a comment - Will be in next release (2.3?)
            abayer Andrew Bayer made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            sergiuionescu Sergiu Ionescu added a comment - - edited

            I can confirm the issue still occurs with version 2.2 and Jenkins ver. 2.91. Do you have an ETA for the next release?

             

            Thanks.

            sergiuionescu Sergiu Ionescu added a comment - - edited I can confirm the issue still occurs with version 2.2 and Jenkins ver. 2.91. Do you have an ETA for the next release?   Thanks.
            sergiuionescu Sergiu Ionescu made changes -
            Environment Jenkins 1.642.4.2 with Cloudbees Enterprise 15.11
            Lockable Resource plugin 1.8
            Jenkins 1.642.4.2 with Cloudbees Enterprise 15.11
            Lockable Resource plugin 1.8
            Lockable Resource plugin 2.2
            dnusbaum Devin Nusbaum made changes -
            Released As lockable-resources 2.3
            bassem bassem ben abbes made changes -
            Rank Ranked higher

            People

              abayer Andrew Bayer
              xvdessel Xavier Van Dessel
              Votes:
              17 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: