Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55485

Declarative pipeline, lock() in stage options is executed before when clause

      In a a declarative pipeline i have a stage grouping several other stages and need to hold a lock for the duration of the group.  I see I can add lock() to the options of the group and it works fine in that scenario.  However the group also has a When clause so I can turn that group off with a job parameter.  

      The problem is that when the "when" evaluates false so the stage will be skipped, it wait for the lock anyway.  If the stage will be skipped for any reason, we should not block on the lock.  The "when" defines a "beforeAgent" attribute for a similar need, so that might be an approach, but I'm not seeing a scenario where you would want to do it in the current order.

       

      My pipeline looks like:

      pipeline {

      agent none
      stages {

            stage ('Build')

      {      }

            stage('Deploy-and-Test') { // Group Stage Deploy and test the Build - lock the deployment VM so we don't step on other runs
                when {
                     expression {
                          "${params.Deploy}" == "true"
                     }
                    beforeAgent true
                }
                options {
                     lock(quantity: 1, resource: "${params.DeployAgent}", variable: 'myDeployAgent')
                }
                agent none

                stages { ...}

           }}}

       

      When I run it withthe param to disable the stage I see:


      [Pipeline] stage[Pipeline] { (Deploy-and-Test)[Pipeline] lockTrying to acquire lock on [DeploymentVM_Win]
      Lock acquired on [DeploymentVM_Win][Pipeline] {Stage "Deploy-and-Test" skipped due to when conditional

      Since the stage is being skipped there is no point in holding the lock, but it the lock is already allocated, this will block anyway.  The lock() should be processed after the decision whether to execute the stage.

          [JENKINS-55485] Declarative pipeline, lock() in stage options is executed before when clause

          D Pasto added a comment -

          I verified the workaround of adding an additional level of stage - I have a group stage with the when clause containing a group stage with the lock, containing the actual stages.  With this structure I'm able to use the when clause to disable deploy and test stages, and not get stuck waiting for a lock behind a long-running test.

          The workaround is conceptually easy, but adds a lot of boilerplate and makes the Jenkinsfile that much harder to read.  I'm now up to 10 levels of tabs in my Jenkinsfile and have a check-in with hundreds of lines of changes that is just indenting.  It's getting pretty ugly.

          This is exactly the mess the "beforeAgent" directive to the "when" clause prevents - we need a similar solution here.

          D Pasto added a comment - I verified the workaround of adding an additional level of stage - I have a group stage with the when clause containing a group stage with the lock, containing the actual stages.  With this structure I'm able to use the when clause to disable deploy and test stages, and not get stuck waiting for a lock behind a long-running test. The workaround is conceptually easy, but adds a lot of boilerplate and makes the Jenkinsfile that much harder to read.  I'm now up to 10 levels of tabs in my Jenkinsfile and have a check-in with hundreds of lines of changes that is just indenting.  It's getting pretty ugly. This is exactly the mess the "beforeAgent" directive to the "when" clause prevents - we need a similar solution here.

          Andy Mrichko added a comment -

          Also faced this issue and want to add that it is really painful in case of concurrent builds that use the same lock resource.

          For example:

          Build 1::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == true ] -----------> Stage 3

          Build 2::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == false ] -----------> Stage 3

           

          Expected behaviour:

          Build 2 skips Stage 2 because it executed not from master branch. Lock not even tried to be allocated

           

          Current  behaviour:

          Build 2 is trying to allocate lock A before checking when conditional, so it will skip Stage 2 only after Stage 2 will be finished in Build 1

          Andy Mrichko added a comment - Also faced this issue and want to add that it is really painful in case of concurrent builds that use the same lock resource. For example: Build 1::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == true ] -----------> Stage 3 Build 2::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == false ] -----------> Stage 3   Expected behaviour: Build 2 skips Stage 2 because it executed not from master branch. Lock not even tried to be allocated   Current  behaviour: Build 2 is trying to allocate lock A before checking when conditional, so it will skip Stage 2 only after Stage 2 will be finished in Build 1

          Is this a duplicate of 51865?

          Christopher Head added a comment - Is this a duplicate of 51865?

          D Pasto added a comment -

          Yes, this duplicates:  JENKINS-51865

          D Pasto added a comment - Yes, this duplicates:   JENKINS-51865

          Falko Modler added a comment - - edited

          Note: JENKINS-51865 is done, new beforeOptions in when is available in pipeline-model-definition 1.4.0.

          Falko Modler added a comment - - edited Note: JENKINS-51865 is done, new beforeOptions in when is available in pipeline-model-definition 1.4.0.

            Unassigned Unassigned
            wgc123 D Pasto
            Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: