Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34268

Lock multiple resources using the Pipeline lock step

      The current implementation of Pipeline lock step allows to block a single resource.

      It should be extended to cover all the functionality of the plugin (applicable to non-freestyle jobs) such as blocking resources by label or request a lock for N resources.

      The DSL must be something like this:

      lock (resources: ['resource1', 'resource2']) {
        ... execution block ...
      }
      

      or

      lock (label: 'my-resources') {
        ... execution block ...
      }
      

      The behavior of the label parameter would be equivalent to:

      lock (resources: ['resource3', 'resource4']) { // if both resource3 and resource4 are labeled as 'my-resources'
        ... execution block ...
      }
      

          [JENKINS-34268] Lock multiple resources using the Pipeline lock step

          amuniz
          Hi! Nice to see this merged.
          Looking at the merged content, I can't seem to find any pipeline syntax for actually knowing what resource was locked.
          Specifically the proposed syntax from PR36 above: ", variable: 'MY_VAR'" is missing.
          Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We wan't to lock a resources (host name) from he label and then ssh to that machine any execute some build steps. This works with the FreestyleJob functionality of #Reserved resources variable name". Any thought of adding this to pipeline?

          Staffan Forsell added a comment - amuniz Hi! Nice to see this merged. Looking at the merged content, I can't seem to find any pipeline syntax for actually knowing what resource was locked. Specifically the proposed syntax from PR36 above: ", variable: 'MY_VAR'" is missing. Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We wan't to lock a resources (host name) from he label and then ssh to that machine any execute some build steps. This works with the FreestyleJob functionality of #Reserved resources variable name". Any thought of adding this to pipeline?

          Karl Schulz added a comment -

          I'm also unable to successfully use the MY_VAR option. It's awesome to be able to lock multiple resources now, but can't recreate the capability of a Freestyle job without being able to query the results.

          Karl Schulz added a comment - I'm also unable to successfully use the MY_VAR option. It's awesome to be able to lock multiple resources now, but can't recreate the capability of a Freestyle job without being able to query the results.

          Anton Lundin added a comment -

          I did a really ugly workaround for the missing resourceVariable :

          lock(label: 'LABEL', quantity: 1) {
          echo org.jenkins.plugins.lockableresources.LockableResourcesManager.class.get().getResourcesFromBuild(currentBuild.getRawBuild())[0].getName()
          }

          That will get you which resource you locked.

          Anton Lundin added a comment - I did a really ugly workaround for the missing resourceVariable : lock(label: 'LABEL', quantity: 1) { echo org.jenkins.plugins.lockableresources.LockableResourcesManager.class.get().getResourcesFromBuild(currentBuild.getRawBuild()) [0] .getName() } That will get you which resource you locked.

          Dan Falcone added a comment - - edited

          After implementing Anton Lundin's suggestion, I received the following error in my build (followed by a stacktrace):

          org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method

          To fix it, I went to Manage Jenkins > In process Script Approval, clicked Approve on the pending signature approval, and reran the build.  I had to repeat the process 4-5 times to approve all the required signatures, but it worked great after that.

          Dan Falcone added a comment - - edited After implementing Anton Lundin's suggestion, I received the following error in my build (followed by a stacktrace): org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method To fix it, I went to Manage Jenkins > In process Script Approval, clicked Approve on the pending signature approval, and reran the build.  I had to repeat the process 4-5 times to approve all the required signatures, but it worked great after that.

          Michael Mrozek added a comment - - edited

          Signatures required for the above workaround:

          method org.jenkins.plugins.lockableresources.LockableResource getName
          method org.jenkins.plugins.lockableresources.LockableResourcesManager getResourcesFromBuild hudson.model.Run
          method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild
          staticMethod org.jenkins.plugins.lockableresources.LockableResourcesManager get

          Michael Mrozek added a comment - - edited Signatures required for the above workaround : method org.jenkins.plugins.lockableresources.LockableResource getName method org.jenkins.plugins.lockableresources.LockableResourcesManager getResourcesFromBuild hudson.model.Run method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild staticMethod org.jenkins.plugins.lockableresources.LockableResourcesManager get

          Mirek Sz added a comment -

          glance

          I did a really ugly workaround for the missing resourceVariable :

          Did you happen to find any way of getting the resource that's locked in the current lock closure? Your workaround always returns the name of first locked resource. I'm trying to run label-based locks in parallel, so I cannot simply use getResourcesFromBuild(...)[0], but must find out the exact resource locked within current scope. Any ideas?

          Mirek Sz added a comment - glance I did a really ugly workaround for the missing resourceVariable : Did you happen to find any way of getting the resource that's locked in the current lock closure? Your workaround always returns the name of first locked resource. I'm trying to run label-based locks in parallel, so I cannot simply use getResourcesFromBuild(...) [0] , but must find out the exact resource locked within current scope. Any ideas?

          Anton Lundin added a comment -

          The hack grew quite a bit over the following days, and ended up as:

          https://gist.github.com/glance-/aaa3c037757895798d4e1be5134bb843

           

          I removed the need for extensive whitelisting by writing it as a pipeline library which later could be imported into the pipeline dsl by:
          @Library("PipelineHelpers")
          import PipelineHelpers.LockableResourcesHelper

           

          We never use different locked resources in parallel so I have never needed to figure out if you can map locked resource to which block that locked it. If you ever do, please post the solution here for others to find.

          Anton Lundin added a comment - The hack grew quite a bit over the following days, and ended up as: https://gist.github.com/glance-/aaa3c037757895798d4e1be5134bb843   I removed the need for extensive whitelisting by writing it as a pipeline library which later could be imported into the pipeline dsl by: @Library("PipelineHelpers") import PipelineHelpers.LockableResourcesHelper   We never use different locked resources in parallel so I have never needed to figure out if you can map locked resource to which block that locked it. If you ever do, please post the solution here for others to find.

          Alan Braggins added a comment - - edited

          We use locked resources in parallel, so the getResourcesFromBuild workaround didn't work for us.

          https://github.com/jenkinsci/lockable-resources-plugin/pull/50 seems to work. (I haven't tried pull/49, but it appears to be a similar approach to the same fix.)

          Alan Braggins added a comment - - edited We use locked resources in parallel, so the getResourcesFromBuild workaround didn't work for us. https://github.com/jenkinsci/lockable-resources-plugin/pull/50  seems to work. (I haven't tried pull/49, but it appears to be a similar approach to the same fix.)

          Anyone hoping to make use of locking a dynamic list of resources will probably find this feature (if it was ever implemented) doesn't work in v2.4 of the plugin.

          So, I whipped up the following helper function which should allow you to do so:

           

          def lockResources(listOfResources, closure) {
              if (listOfResources.size() > 1) {
                  lockResources(listOfResources[1..-1], { lock(listOfResources[0]) { closure() }})
              } else {
                  lock(listOfResources[0]) { closure() }
              }
          }
          

          Example usage:

          def resources = ['foo', 'bar', 'baz']
          lockResources(myResources, { 
              print('hello, world') 
          })
          

           

           

          Chris Dickinson added a comment - Anyone hoping to make use of locking a dynamic list of resources will probably find this feature (if it was ever implemented) doesn't work in v2.4 of the plugin. So, I whipped up the following helper function which should allow you to do so:   def lockResources(listOfResources, closure) { if (listOfResources.size() > 1) { lockResources(listOfResources[1..-1], { lock(listOfResources[0]) { closure() }}) } else { lock(listOfResources[0]) { closure() } } } Example usage: def resources = [ 'foo' , 'bar' , 'baz' ] lockResources(myResources, { print( 'hello, world' ) })    

          chrisdickinsonnio Can you open a new ticket instead of commenting on a closed one (where nobody will ever see your comments, I stumbled on it by accident) if you have issues with the plugin? Anyway, locking of multiple resources is implemented since 2.3. The syntax (as generated by the snippet generator) is:

          lock(extra: [[resource: 'b'], [label: 'c', quantity: 1]], resource: 'a') {
              // some block
          }
          

           

          Tobias Gruetzmacher added a comment - chrisdickinsonnio Can you open a new ticket instead of commenting on a closed one (where nobody will ever see your comments, I stumbled on it by accident) if you have issues with the plugin? Anyway, locking of multiple resources is implemented since 2.3. The syntax (as generated by the snippet generator) is: lock(extra: [[resource: 'b' ], [label: 'c' , quantity: 1]], resource: 'a' ) { // some block }  

            amuniz Antonio Muñiz
            amuniz Antonio Muñiz
            Votes:
            19 Vote for this issue
            Watchers:
            33 Start watching this issue

              Created:
              Updated:
              Resolved: