• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • None
    • lockable-resources 1.11

      https://issues.jenkins-ci.org/browse/JENKINS-34268 was closed implementing the possibility to lock a resources using a label in pipelines.
      But there is no syntax for knowing what the actually locked resource name is when using a pipeline and locking on a label.
      Specifically the proposed syntax from https://github.com/jenkinsci/lockable-resources-plugin/pull/36 : ", variable: 'MY_VAR'" is missing.

      Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We want to lock a resource (host name) from the label and then ssh to that machine and execute some build steps. This works with the FreestyleJob functionality of "Reserved resources variable name".

      Solution

      It is now possible to access the locked resource via an configurable environment variable.

      lock(label: 'some_resource', variable: 'MY_VAR') {
        echo env.MY_VAR
      }
      

       

       

          [JENKINS-40997] Add way to get locked resource name in pipeline

          Wouldn't more useful to provide a map of environment variables for each resource?  This way the complete parameterization can be done in one place (configuration). Otherwise you have to code some conditions based on resource name in your pipeline. (we do it now using Anton Lundin's workaround)

          Alexander Kern added a comment - Wouldn't more useful to provide a map of environment variables for each resource?  This way the complete parameterization can be done in one place (configuration). Otherwise you have to code some conditions based on resource name in your pipeline. (we do it now using Anton Lundin's workaround)

          Daniel Geißler added a comment - - edited

          Anton Lundins workaround has some inconveniences. If you for example use multiple locks stacked on top of each other, you will not get the resource locked by the individual step, but all that the currentBuild is holding.

          You can work around this but it may be more convinient to actually get the lock that was reserved in the current call.

          Personally I am even more into having non block scoped lock steps with a groovy based return value (e.g. a map would be useful when locking multiple resources with a label). This is already described in https://github.com/jenkinsci/lockable-resources-plugin/pull/64

           

          Daniel Geißler added a comment - - edited Anton Lundins workaround has some inconveniences. If you for example use multiple locks stacked on top of each other, you will not get the resource locked by the individual step, but all that the currentBuild is holding. You can work around this but it may be more convinient to actually get the lock that was reserved in the current call. Personally I am even more into having non block scoped lock steps with a groovy based return value (e.g. a map would be useful when locking multiple resources with a label). This is already described in https://github.com/jenkinsci/lockable-resources-plugin/pull/64  

          I'm not voting for workaround (I use it, because it works), and I would really enjoy proper realization.

           

          Multiple stacking is not a problem in case of using environment as I wrote above. In rare case anybody really need a name of resource, it can be exported as resource object  addressed by 'resource label'. Even the list of resources can be realized as list of objects in case of multiple locking. 'Name', 'Description' and etc. can be some attributes of it.

          Personally I'm prefer scoped lock. If something can be forgotten, humans forget this.

           

           

          Alexander Kern added a comment - I'm not voting for workaround (I use it, because it works), and I would really enjoy proper realization.   Multiple stacking is not a problem in case of using environment as I wrote above. In rare case anybody really need a name of resource, it can be exported as resource object  addressed by 'resource label'. Even the list of resources can be realized as list of objects in case of multiple locking. 'Name', 'Description' and etc. can be some attributes of it. Personally I'm prefer scoped lock. If something can be forgotten, humans forget this.    

          Mirek Sz added a comment - - edited

          kernalex: I cannot think of an idea how such "each resource to variable name" mapping would work. Consider following scenario:

          We have 3 resources, two labels:

          • resource_1, with label_A, mapping to variable X
          • resource_2, with label_B, mapping to variable Y
          • resource_3, with label_A, mapping to variable Z

          Now you lock resource with label_A - under which variable name should I find the resource name? X or Z?

          On the other hand, mapping label to variable name wouldn't work as well - let's say I lock one resource of label_A, the resource name lands in variable A, than a while later I lock another resource of label_A - and this time I'm overriding the A variable with another resource name.

          If you want to keep your resource configuration apart from using it, I suggest creating the mappings yourself at the very beginning of your script and passing them as parameter to lock():

          def my_resource = [label: 'your_resource_label', variable: 'your_variable_name']
          ...
          // later
          ...
          lock(my_resource) {
            ...
          }

          As for multiple stacks, you could define a specific class that would return the mapping [label: 'your_resource_label', variable: 'variable_' + i++], every time you call it:

          class my_resource_class {
              static i = 1
              def variable, label
              my_resource_class(label, variable) {
                  this.variable = variable
                  this.label = label
              }
              def call() {
                  return [label: label, variable: variable + i++]
              }
          }
          my_resource = new my_resource_class('your_label', 'variable_')
          
          lock (my_resource()) {    // first call resolves to [...variable: "variable_1"]
            lock (my_resource()) {  // second call to [...variable: "variable_2"] 
              ...
            }
          }
          

          Mirek Sz added a comment - - edited kernalex : I cannot think of an idea how such " each resource to variable name " mapping would work. Consider following scenario: We have 3 resources, two labels: resource_1, with label_A, mapping to variable X resource_2, with label_B, mapping to variable Y resource_3, with label_A, mapping to variable Z Now you lock resource with label_A - under which variable name should I find the resource name? X or Z? On the other hand, mapping label to variable name wouldn't work as well - let's say I lock one resource of label_A, the resource name lands in variable A, than a while later I lock another resource of label_A - and this time I'm overriding the A variable with another resource name. If you want to keep your resource configuration apart from using it, I suggest creating the mappings yourself at the very beginning of your script and passing them as parameter to lock(): def my_resource = [label: 'your_resource_label' , variable: 'your_variable_name' ] ... // later ... lock(my_resource) { ... } As for multiple stacks, you could define a specific class that would return the mapping [label: 'your_resource_label', variable: 'variable_' + i++] , every time you call it: class my_resource_class { static i = 1 def variable, label my_resource_class(label, variable) { this .variable = variable this .label = label } def call() { return [label: label, variable: variable + i++] } } my_resource = new my_resource_class( 'your_label' , 'variable_' ) lock (my_resource()) { // first call resolves to [...variable: "variable_1" ] lock (my_resource()) { // second call to [...variable: "variable_2" ] ... } }

          Hi,

           

          I would agree with you. Automatic mapping "resource to variable" will be ambiguous in any case.

           

          My concern is more about ability to provide environment variables only in configuration.

          Alexander Kern added a comment - Hi,   I would agree with you. Automatic mapping "resource to variable" will be ambiguous in any case.   My concern is more about ability to provide environment variables only in configuration.

          Julian F. added a comment -

          Julian F. added a comment - Implemented with https://github.com/jenkinsci/lockable-resources-plugin/pull/49

          Released as 2.2

          Antonio Muñiz added a comment - Released as 2.2

          Brandon Saunders added a comment - - edited

          I'm appearing to hit an issue where a released resource does not get an update on the environment variable once the lock is released.  See the following code (with only two resources in the automation-accounts label)

           

          node {
            parallel (
              "p1": {
                lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {
                  echo "A $env.ACCOUNTS_VAR"
                  sleep 4
                       }
              },
              "p2": {
                lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {
                  echo "B $env.ACCOUNTS_VAR"
                  sleep 2
                       }
              },
              "p3": {
                lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {
                  echo "C $env.ACCOUNTS_VAR"
                       }
                   }
            )
          }

           

          In the log I get the following (note "C null"):
          [Pipeline] {
          [Pipeline] parallel
          [Pipeline] [p1] { (Branch: p1)
          [Pipeline] [p2] { (Branch: p2)
          [Pipeline] [p3] { (Branch: p3)
          [Pipeline] [p1] lock
          [p1] Trying to acquire lock on [Label: automation-accounts, Quantity: 1]
          [p1] Lock acquired on [Label: automation-accounts, Quantity: 1]
          [Pipeline] [p1] {
          [Pipeline] [p2] lock
          [p2] Trying to acquire lock on [Label: automation-accounts, Quantity: 1]
          [p2] Lock acquired on [Label: automation-accounts, Quantity: 1]
          [Pipeline] [p2] {
          [Pipeline] [p3] lock
          [p3] Trying to acquire lock on [Label: automation-accounts, Quantity: 1]
          [p3] Found 0 available resource(s). Waiting for correct amount: 1.
          [p3] [Label: automation-accounts, Quantity: 1] is locked, waiting...
          [Pipeline] [p1] echo
          [p1] A <Account1>
          [Pipeline] [p1] sleep
          [p1] Sleeping for 4 sec
          [Pipeline] [p2] echo
          [p2] B <Account2>
          [Pipeline] [p2] sleep
          [p2] Sleeping for 2 sec
          [p3] Lock acquired on [Label: automation-accounts, Quantity: 1]
          [Pipeline] [p2] }
          [p2] Lock released on resource [Label: automation-accounts, Quantity: 1]
          [Pipeline] [p3] {
          [Pipeline] [p2] // lock
          [Pipeline] [p2] }
          [Pipeline] [p3] echo
          [p3] C null
          [Pipeline] [p3] }
          [p3] Lock released on resource [Label: automation-accounts, Quantity: 1]
          [Pipeline] [p3] // lock
          [Pipeline] [p3] }
          [Pipeline] [p1] }
          [p1] Lock released on resource [Label: automation-accounts, Quantity: 1]
          [Pipeline] [p1] // lock
          [Pipeline] [p1] }
          [Pipeline] // parallel
          [Pipeline] }
          [Pipeline] // node
          [Pipeline] End of Pipeline

          Expected Result:
          I should get "C <Account2>

          Perhaps I'm calling it wrong though?

          Brandon Saunders added a comment - - edited I'm appearing to hit an issue where a released resource does not get an update on the environment variable once the lock is released.  See the following code (with only two resources in the automation-accounts label)   node {   parallel (     "p1": {       lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {         echo "A $env.ACCOUNTS_VAR"         sleep 4              }     },     "p2": {       lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {         echo "B $env.ACCOUNTS_VAR"         sleep 2              }     },     "p3": {       lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {          echo "C $env.ACCOUNTS_VAR"              }          }   ) }   In the log I get the following (note "C null"): [Pipeline] { [Pipeline] parallel [Pipeline] [p1] { (Branch: p1) [Pipeline] [p2] { (Branch: p2) [Pipeline] [p3] { (Branch: p3) [Pipeline] [p1] lock [p1] Trying to acquire lock on [Label: automation-accounts, Quantity: 1] [p1] Lock acquired on [Label: automation-accounts, Quantity: 1] [Pipeline] [p1] { [Pipeline] [p2] lock [p2] Trying to acquire lock on [Label: automation-accounts, Quantity: 1] [p2] Lock acquired on [Label: automation-accounts, Quantity: 1] [Pipeline] [p2] { [Pipeline] [p3] lock [p3] Trying to acquire lock on [Label: automation-accounts, Quantity: 1] [p3] Found 0 available resource(s). Waiting for correct amount: 1. [p3] [Label: automation-accounts, Quantity: 1] is locked, waiting... [Pipeline] [p1] echo [p1] A <Account1> [Pipeline] [p1] sleep [p1] Sleeping for 4 sec [Pipeline] [p2] echo [p2] B <Account2> [Pipeline] [p2] sleep [p2] Sleeping for 2 sec [p3] Lock acquired on [Label: automation-accounts, Quantity: 1] [Pipeline] [p2] } [p2] Lock released on resource [Label: automation-accounts, Quantity: 1] [Pipeline] [p3] { [Pipeline] [p2] // lock [Pipeline] [p2] } [Pipeline] [p3] echo [p3] C null [Pipeline] [p3] } [p3] Lock released on resource [Label: automation-accounts, Quantity: 1] [Pipeline] [p3] // lock [Pipeline] [p3] } [Pipeline] [p1] } [p1] Lock released on resource [Label: automation-accounts, Quantity: 1] [Pipeline] [p1] // lock [Pipeline] [p1] } [Pipeline] // parallel [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Expected Result : I should get "C <Account2> Perhaps I'm calling it wrong though?

          Hi All,

          Same behavior as Brandon's happens to me:

          I have several jobs requiring a specific resource, and I only have 4 of them property labeled.

          If at the moment of lock there is no resources available pipeline will wait until resource gets free

          Once is available, pipeline resumes and effectively reserves the resource but variable does not get filled. (Label value), instead gets null value.

          Regards

          Alfredo

          Alfredo Castro added a comment - Hi All, Same behavior as Brandon's happens to me: I have several jobs requiring a specific resource, and I only have 4 of them property labeled. If at the moment of lock there is no resources available pipeline will wait until resource gets free Once is available, pipeline resumes and effectively reserves the resource but variable does not get filled. (Label value), instead gets null value. Regards Alfredo

          Jeroen Bogers added a comment - - edited

          EDIT: See my comment on JENKINS-50176

          Jeroen Bogers added a comment - - edited EDIT: See my comment on JENKINS-50176

            f_julian Julian F.
            fkykko Staffan Forsell
            Votes:
            24 Vote for this issue
            Watchers:
            27 Start watching this issue

              Created:
              Updated:
              Resolved: