Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55485

Declarative pipeline, lock() in stage options is executed before when clause

    XMLWordPrintable

Details

    Description

      In a a declarative pipeline i have a stage grouping several other stages and need to hold a lock for the duration of the group.  I see I can add lock() to the options of the group and it works fine in that scenario.  However the group also has a When clause so I can turn that group off with a job parameter.  

      The problem is that when the "when" evaluates false so the stage will be skipped, it wait for the lock anyway.  If the stage will be skipped for any reason, we should not block on the lock.  The "when" defines a "beforeAgent" attribute for a similar need, so that might be an approach, but I'm not seeing a scenario where you would want to do it in the current order.

       

      My pipeline looks like:

      pipeline {

      agent none
      stages {

            stage ('Build')

      {      }

            stage('Deploy-and-Test') { // Group Stage Deploy and test the Build - lock the deployment VM so we don't step on other runs
                when {
                     expression {
                          "${params.Deploy}" == "true"
                     }
                    beforeAgent true
                }
                options {
                     lock(quantity: 1, resource: "${params.DeployAgent}", variable: 'myDeployAgent')
                }
                agent none

                stages { ...}

           }}}

       

      When I run it withthe param to disable the stage I see:


      [Pipeline] stage[Pipeline] { (Deploy-and-Test)[Pipeline] lockTrying to acquire lock on [DeploymentVM_Win]
      Lock acquired on [DeploymentVM_Win][Pipeline] {Stage "Deploy-and-Test" skipped due to when conditional

      Since the stage is being skipped there is no point in holding the lock, but it the lock is already allocated, this will block anyway.  The lock() should be processed after the decision whether to execute the stage.

      Attachments

        Issue Links

          Activity

            wgc123 D Pasto added a comment -

            I verified the workaround of adding an additional level of stage - I have a group stage with the when clause containing a group stage with the lock, containing the actual stages.  With this structure I'm able to use the when clause to disable deploy and test stages, and not get stuck waiting for a lock behind a long-running test.

            The workaround is conceptually easy, but adds a lot of boilerplate and makes the Jenkinsfile that much harder to read.  I'm now up to 10 levels of tabs in my Jenkinsfile and have a check-in with hundreds of lines of changes that is just indenting.  It's getting pretty ugly.

            This is exactly the mess the "beforeAgent" directive to the "when" clause prevents - we need a similar solution here.

            wgc123 D Pasto added a comment - I verified the workaround of adding an additional level of stage - I have a group stage with the when clause containing a group stage with the lock, containing the actual stages.  With this structure I'm able to use the when clause to disable deploy and test stages, and not get stuck waiting for a lock behind a long-running test. The workaround is conceptually easy, but adds a lot of boilerplate and makes the Jenkinsfile that much harder to read.  I'm now up to 10 levels of tabs in my Jenkinsfile and have a check-in with hundreds of lines of changes that is just indenting.  It's getting pretty ugly. This is exactly the mess the "beforeAgent" directive to the "when" clause prevents - we need a similar solution here.
            amrichko Andy Mrichko added a comment -

            Also faced this issue and want to add that it is really painful in case of concurrent builds that use the same lock resource.

            For example:

            Build 1::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == true ] -----------> Stage 3

            Build 2::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == false ] -----------> Stage 3

             

            Expected behaviour:

            Build 2 skips Stage 2 because it executed not from master branch. Lock not even tried to be allocated

             

            Current  behaviour:

            Build 2 is trying to allocate lock A before checking when conditional, so it will skip Stage 2 only after Stage 2 will be finished in Build 1

            amrichko Andy Mrichko added a comment - Also faced this issue and want to add that it is really painful in case of concurrent builds that use the same lock resource. For example: Build 1::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == true ] -----------> Stage 3 Build 2::  Stage 1 ----------> Stage 2 (get lock A) [(when branch is master) == false ] -----------> Stage 3   Expected behaviour: Build 2 skips Stage 2 because it executed not from master branch. Lock not even tried to be allocated   Current  behaviour: Build 2 is trying to allocate lock A before checking when conditional, so it will skip Stage 2 only after Stage 2 will be finished in Build 1

            Is this a duplicate of 51865?

            haw777 Christopher Head added a comment - Is this a duplicate of 51865?
            wgc123 D Pasto added a comment -

            Yes, this duplicates:  JENKINS-51865

            wgc123 D Pasto added a comment - Yes, this duplicates:   JENKINS-51865
            famod Falko Modler added a comment - - edited

            Note: JENKINS-51865 is done, new beforeOptions in when is available in pipeline-model-definition 1.4.0.

            famod Falko Modler added a comment - - edited Note: JENKINS-51865 is done, new beforeOptions in when is available in pipeline-model-definition 1.4.0.

            People

              Unassigned Unassigned
              wgc123 D Pasto
              Votes:
              3 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: