Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40997

Add way to get locked resource name in pipeline

    XMLWordPrintable

Details

    Description

      https://issues.jenkins-ci.org/browse/JENKINS-34268 was closed implementing the possibility to lock a resources using a label in pipelines.
      But there is no syntax for knowing what the actually locked resource name is when using a pipeline and locking on a label.
      Specifically the proposed syntax from https://github.com/jenkinsci/lockable-resources-plugin/pull/36 : ", variable: 'MY_VAR'" is missing.

      Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We want to lock a resource (host name) from the label and then ssh to that machine and execute some build steps. This works with the FreestyleJob functionality of "Reserved resources variable name".

      Solution

      It is now possible to access the locked resource via an configurable environment variable.

      lock(label: 'some_resource', variable: 'MY_VAR') {
        echo env.MY_VAR
      }
      

       

       

      Attachments

        Issue Links

          Activity

            fkykko Staffan Forsell created issue -
            dageissl Daniel Geißler added a comment - Just to link the workaround from Anton Lundin as long as it is not implemented: https://issues.jenkins-ci.org/browse/JENKINS-34268?focusedCommentId=281694&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-281694
            f_julian Julian F. added a comment - - edited

            Hi i tried to implement this with https://github.com/jenkinsci/lockable-resources-plugin/pull/50

            Maybe someone can have a look?

            f_julian Julian F. added a comment - - edited Hi i tried to implement this with https://github.com/jenkinsci/lockable-resources-plugin/pull/50 Maybe someone can have a look?
            f_julian Julian F. made changes -
            Field Original Value New Value
            Assignee Julian F. [ f_julian ]
            f_julian Julian F. made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            f_julian Julian F. made changes -
            Description https://issues.jenkins-ci.org/browse/JENKINS-34268 was closed implementing the possibility to lock a resources using a label in pipelines.
            But there is no syntax for knowing what the actually locked resource name is when using a pipeline and locking on a label.
            Specifically the proposed syntax from https://github.com/jenkinsci/lockable-resources-plugin/pull/36 : ", variable: 'MY_VAR'" is missing.

            Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We wan't to lock a resource (host name) from the label and then ssh to that machine and execute some build steps. This works with the FreestyleJob functionality of "Reserved resources variable name".
            https://issues.jenkins-ci.org/browse/JENKINS-34268 was closed implementing the possibility to lock a resources using a label in pipelines.
             But there is no syntax for knowing what the actually locked resource name is when using a pipeline and locking on a label.
             Specifically the proposed syntax from [https://github.com/jenkinsci/lockable-resources-plugin/pull/36] : ", variable: 'MY_VAR'" is missing.

            Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We wan't to lock a resource (host name) from the label and then ssh to that machine and execute some build steps. This works with the FreestyleJob functionality of "Reserved resources variable name".
            h4. Solution

            It is now possible to access the locked resource via an configurable environment variable.
            {code:java}
            lock(label: 'some_resource', variable: 'MY_VAR') {
              echo env.MY_VAR
            }
            {code}
             

             
            f_julian Julian F. made changes -
            Remote Link This issue links to " PR #50 (Web Link)" [ 15835 ]

            Wouldn't more useful to provide a map of environment variables for each resource?  This way the complete parameterization can be done in one place (configuration). Otherwise you have to code some conditions based on resource name in your pipeline. (we do it now using Anton Lundin's workaround)

            kernalex Alexander Kern added a comment - Wouldn't more useful to provide a map of environment variables for each resource?  This way the complete parameterization can be done in one place (configuration). Otherwise you have to code some conditions based on resource name in your pipeline. (we do it now using Anton Lundin's workaround)
            dageissl Daniel Geißler added a comment - - edited

            Anton Lundins workaround has some inconveniences. If you for example use multiple locks stacked on top of each other, you will not get the resource locked by the individual step, but all that the currentBuild is holding.

            You can work around this but it may be more convinient to actually get the lock that was reserved in the current call.

            Personally I am even more into having non block scoped lock steps with a groovy based return value (e.g. a map would be useful when locking multiple resources with a label). This is already described in https://github.com/jenkinsci/lockable-resources-plugin/pull/64

             

            dageissl Daniel Geißler added a comment - - edited Anton Lundins workaround has some inconveniences. If you for example use multiple locks stacked on top of each other, you will not get the resource locked by the individual step, but all that the currentBuild is holding. You can work around this but it may be more convinient to actually get the lock that was reserved in the current call. Personally I am even more into having non block scoped lock steps with a groovy based return value (e.g. a map would be useful when locking multiple resources with a label). This is already described in https://github.com/jenkinsci/lockable-resources-plugin/pull/64  

            I'm not voting for workaround (I use it, because it works), and I would really enjoy proper realization.

             

            Multiple stacking is not a problem in case of using environment as I wrote above. In rare case anybody really need a name of resource, it can be exported as resource object  addressed by 'resource label'. Even the list of resources can be realized as list of objects in case of multiple locking. 'Name', 'Description' and etc. can be some attributes of it.

            Personally I'm prefer scoped lock. If something can be forgotten, humans forget this.

             

             

            kernalex Alexander Kern added a comment - I'm not voting for workaround (I use it, because it works), and I would really enjoy proper realization.   Multiple stacking is not a problem in case of using environment as I wrote above. In rare case anybody really need a name of resource, it can be exported as resource object  addressed by 'resource label'. Even the list of resources can be realized as list of objects in case of multiple locking. 'Name', 'Description' and etc. can be some attributes of it. Personally I'm prefer scoped lock. If something can be forgotten, humans forget this.    
            mireksz Mirek Sz added a comment - - edited

            kernalex: I cannot think of an idea how such "each resource to variable name" mapping would work. Consider following scenario:

            We have 3 resources, two labels:

            • resource_1, with label_A, mapping to variable X
            • resource_2, with label_B, mapping to variable Y
            • resource_3, with label_A, mapping to variable Z

            Now you lock resource with label_A - under which variable name should I find the resource name? X or Z?

            On the other hand, mapping label to variable name wouldn't work as well - let's say I lock one resource of label_A, the resource name lands in variable A, than a while later I lock another resource of label_A - and this time I'm overriding the A variable with another resource name.

            If you want to keep your resource configuration apart from using it, I suggest creating the mappings yourself at the very beginning of your script and passing them as parameter to lock():

            def my_resource = [label: 'your_resource_label', variable: 'your_variable_name']
            ...
            // later
            ...
            lock(my_resource) {
              ...
            }

            As for multiple stacks, you could define a specific class that would return the mapping [label: 'your_resource_label', variable: 'variable_' + i++], every time you call it:

            class my_resource_class {
                static i = 1
                def variable, label
                my_resource_class(label, variable) {
                    this.variable = variable
                    this.label = label
                }
                def call() {
                    return [label: label, variable: variable + i++]
                }
            }
            my_resource = new my_resource_class('your_label', 'variable_')
            
            lock (my_resource()) {    // first call resolves to [...variable: "variable_1"]
              lock (my_resource()) {  // second call to [...variable: "variable_2"] 
                ...
              }
            }
            
            mireksz Mirek Sz added a comment - - edited kernalex : I cannot think of an idea how such " each resource to variable name " mapping would work. Consider following scenario: We have 3 resources, two labels: resource_1, with label_A, mapping to variable X resource_2, with label_B, mapping to variable Y resource_3, with label_A, mapping to variable Z Now you lock resource with label_A - under which variable name should I find the resource name? X or Z? On the other hand, mapping label to variable name wouldn't work as well - let's say I lock one resource of label_A, the resource name lands in variable A, than a while later I lock another resource of label_A - and this time I'm overriding the A variable with another resource name. If you want to keep your resource configuration apart from using it, I suggest creating the mappings yourself at the very beginning of your script and passing them as parameter to lock(): def my_resource = [label: 'your_resource_label' , variable: 'your_variable_name' ] ... // later ... lock(my_resource) { ... } As for multiple stacks, you could define a specific class that would return the mapping [label: 'your_resource_label', variable: 'variable_' + i++] , every time you call it: class my_resource_class { static i = 1 def variable, label my_resource_class(label, variable) { this .variable = variable this .label = label } def call() { return [label: label, variable: variable + i++] } } my_resource = new my_resource_class( 'your_label' , 'variable_' ) lock (my_resource()) { // first call resolves to [...variable: "variable_1" ] lock (my_resource()) { // second call to [...variable: "variable_2" ] ... } }

            Hi,

             

            I would agree with you. Automatic mapping "resource to variable" will be ambiguous in any case.

             

            My concern is more about ability to provide environment variables only in configuration.

            kernalex Alexander Kern added a comment - Hi,   I would agree with you. Automatic mapping "resource to variable" will be ambiguous in any case.   My concern is more about ability to provide environment variables only in configuration.
            f_julian Julian F. made changes -
            Remote Link This issue links to "PR #49 (Web Link)" [ 20098 ]
            f_julian Julian F. made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            f_julian Julian F. added a comment - Implemented with https://github.com/jenkinsci/lockable-resources-plugin/pull/49
            f_julian Julian F. made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]

            Released as 2.2

            amuniz Antonio Muñiz added a comment - Released as 2.2
            avidviewer Brandon Saunders added a comment - - edited

            I'm appearing to hit an issue where a released resource does not get an update on the environment variable once the lock is released.  See the following code (with only two resources in the automation-accounts label)

             

            node {
              parallel (
                "p1": {
                  lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {
                    echo "A $env.ACCOUNTS_VAR"
                    sleep 4
                         }
                },
                "p2": {
                  lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {
                    echo "B $env.ACCOUNTS_VAR"
                    sleep 2
                         }
                },
                "p3": {
                  lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {
                    echo "C $env.ACCOUNTS_VAR"
                         }
                     }
              )
            }

             

            In the log I get the following (note "C null"):
            [Pipeline] {
            [Pipeline] parallel
            [Pipeline] [p1] { (Branch: p1)
            [Pipeline] [p2] { (Branch: p2)
            [Pipeline] [p3] { (Branch: p3)
            [Pipeline] [p1] lock
            [p1] Trying to acquire lock on [Label: automation-accounts, Quantity: 1]
            [p1] Lock acquired on [Label: automation-accounts, Quantity: 1]
            [Pipeline] [p1] {
            [Pipeline] [p2] lock
            [p2] Trying to acquire lock on [Label: automation-accounts, Quantity: 1]
            [p2] Lock acquired on [Label: automation-accounts, Quantity: 1]
            [Pipeline] [p2] {
            [Pipeline] [p3] lock
            [p3] Trying to acquire lock on [Label: automation-accounts, Quantity: 1]
            [p3] Found 0 available resource(s). Waiting for correct amount: 1.
            [p3] [Label: automation-accounts, Quantity: 1] is locked, waiting...
            [Pipeline] [p1] echo
            [p1] A <Account1>
            [Pipeline] [p1] sleep
            [p1] Sleeping for 4 sec
            [Pipeline] [p2] echo
            [p2] B <Account2>
            [Pipeline] [p2] sleep
            [p2] Sleeping for 2 sec
            [p3] Lock acquired on [Label: automation-accounts, Quantity: 1]
            [Pipeline] [p2] }
            [p2] Lock released on resource [Label: automation-accounts, Quantity: 1]
            [Pipeline] [p3] {
            [Pipeline] [p2] // lock
            [Pipeline] [p2] }
            [Pipeline] [p3] echo
            [p3] C null
            [Pipeline] [p3] }
            [p3] Lock released on resource [Label: automation-accounts, Quantity: 1]
            [Pipeline] [p3] // lock
            [Pipeline] [p3] }
            [Pipeline] [p1] }
            [p1] Lock released on resource [Label: automation-accounts, Quantity: 1]
            [Pipeline] [p1] // lock
            [Pipeline] [p1] }
            [Pipeline] // parallel
            [Pipeline] }
            [Pipeline] // node
            [Pipeline] End of Pipeline

            Expected Result:
            I should get "C <Account2>

            Perhaps I'm calling it wrong though?

            avidviewer Brandon Saunders added a comment - - edited I'm appearing to hit an issue where a released resource does not get an update on the environment variable once the lock is released.  See the following code (with only two resources in the automation-accounts label)   node {   parallel (     "p1": {       lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {         echo "A $env.ACCOUNTS_VAR"         sleep 4              }     },     "p2": {       lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {         echo "B $env.ACCOUNTS_VAR"         sleep 2              }     },     "p3": {       lock(label: 'automation-accounts', variable: 'ACCOUNTS_VAR', quantity: 1) {          echo "C $env.ACCOUNTS_VAR"              }          }   ) }   In the log I get the following (note "C null"): [Pipeline] { [Pipeline] parallel [Pipeline] [p1] { (Branch: p1) [Pipeline] [p2] { (Branch: p2) [Pipeline] [p3] { (Branch: p3) [Pipeline] [p1] lock [p1] Trying to acquire lock on [Label: automation-accounts, Quantity: 1] [p1] Lock acquired on [Label: automation-accounts, Quantity: 1] [Pipeline] [p1] { [Pipeline] [p2] lock [p2] Trying to acquire lock on [Label: automation-accounts, Quantity: 1] [p2] Lock acquired on [Label: automation-accounts, Quantity: 1] [Pipeline] [p2] { [Pipeline] [p3] lock [p3] Trying to acquire lock on [Label: automation-accounts, Quantity: 1] [p3] Found 0 available resource(s). Waiting for correct amount: 1. [p3] [Label: automation-accounts, Quantity: 1] is locked, waiting... [Pipeline] [p1] echo [p1] A <Account1> [Pipeline] [p1] sleep [p1] Sleeping for 4 sec [Pipeline] [p2] echo [p2] B <Account2> [Pipeline] [p2] sleep [p2] Sleeping for 2 sec [p3] Lock acquired on [Label: automation-accounts, Quantity: 1] [Pipeline] [p2] } [p2] Lock released on resource [Label: automation-accounts, Quantity: 1] [Pipeline] [p3] { [Pipeline] [p2] // lock [Pipeline] [p2] } [Pipeline] [p3] echo [p3] C null [Pipeline] [p3] } [p3] Lock released on resource [Label: automation-accounts, Quantity: 1] [Pipeline] [p3] // lock [Pipeline] [p3] } [Pipeline] [p1] } [p1] Lock released on resource [Label: automation-accounts, Quantity: 1] [Pipeline] [p1] // lock [Pipeline] [p1] } [Pipeline] // parallel [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Expected Result : I should get "C <Account2> Perhaps I'm calling it wrong though?

            Hi All,

            Same behavior as Brandon's happens to me:

            I have several jobs requiring a specific resource, and I only have 4 of them property labeled.

            If at the moment of lock there is no resources available pipeline will wait until resource gets free

            Once is available, pipeline resumes and effectively reserves the resource but variable does not get filled. (Label value), instead gets null value.

            Regards

            Alfredo

            acastrocuartero Alfredo Castro added a comment - Hi All, Same behavior as Brandon's happens to me: I have several jobs requiring a specific resource, and I only have 4 of them property labeled. If at the moment of lock there is no resources available pipeline will wait until resource gets free Once is available, pipeline resumes and effectively reserves the resource but variable does not get filled. (Label value), instead gets null value. Regards Alfredo
            jbogers Jeroen Bogers added a comment - - edited

            EDIT: See my comment on JENKINS-50176

            jbogers Jeroen Bogers added a comment - - edited EDIT: See my comment on JENKINS-50176
            jbogers Jeroen Bogers made changes -
            Link This issue is related to JENKINS-50176 [ JENKINS-50176 ]
            chrismaes Chris Maes made changes -
            Description https://issues.jenkins-ci.org/browse/JENKINS-34268 was closed implementing the possibility to lock a resources using a label in pipelines.
             But there is no syntax for knowing what the actually locked resource name is when using a pipeline and locking on a label.
             Specifically the proposed syntax from [https://github.com/jenkinsci/lockable-resources-plugin/pull/36] : ", variable: 'MY_VAR'" is missing.

            Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We wan't to lock a resource (host name) from the label and then ssh to that machine and execute some build steps. This works with the FreestyleJob functionality of "Reserved resources variable name".
            h4. Solution

            It is now possible to access the locked resource via an configurable environment variable.
            {code:java}
            lock(label: 'some_resource', variable: 'MY_VAR') {
              echo env.MY_VAR
            }
            {code}
             

             
            https://issues.jenkins-ci.org/browse/JENKINS-34268 was closed implementing the possibility to lock a resources using a label in pipelines.
             But there is no syntax for knowing what the actually locked resource name is when using a pipeline and locking on a label.
             Specifically the proposed syntax from [https://github.com/jenkinsci/lockable-resources-plugin/pull/36] : ", variable: 'MY_VAR'" is missing.

            Our use case is that we wan't to specify a label containing node names for legacy builders that can't run the slave.jar (java7) anymore. We want to lock a resource (host name) from the label and then ssh to that machine and execute some build steps. This works with the FreestyleJob functionality of "Reserved resources variable name".
            h4. Solution

            It is now possible to access the locked resource via an configurable environment variable.
            {code:java}
            lock(label: 'some_resource', variable: 'MY_VAR') {
              echo env.MY_VAR
            }
            {code}
             

             

            People

              f_julian Julian F.
              fkykko Staffan Forsell
              Votes:
              24 Vote for this issue
              Watchers:
              27 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: