Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43336

Allow locking multiple stages in declarative pipeline

    XMLWordPrintable

Details

    • Declarative - 1.2

    Description

      It would be useful to be able to lock multiple stages as a single lock. For example, we usually have a stage to deploy to an environment and then another stage to run end-to-end tests on that environment, but there should be no other concurrent deployments until both stages have completed.

      Something like this:

      pipeline {
        stages {
          lock(resource: 'myResource', inversePrecedence: true) {
            stage('Deploy') {
              // deploy to environment
            }
      
            stage('E2E') {
              // run tests on the environment
              milestone 1
            }
          }
        }
      }

      Technically both stages could just be merged into a single stage but to me that defeats the purpose of stages.

      Attachments

        Issue Links

          Activity

            rrueth Ryan Rueth added a comment -

            abayer, I'm trying to lock multiple stages based on the Jenkins node on which the stages will execute. It seems like the only way to do this is using the `options` block that you specified:

            stage('Parent') {
              options {
                lock("${env.NODE_NAME}")
              }
              stages {
                stage('first child') {
                  ...
                }
                stage('second child') {
                  ...
                }
              }
            }

            Unfortunately, the resource this attempts to lock is `null`:

            Trying to acquire lock on [null]

             

            It appears that this won't work because the stage's `options` directive occurs before entering the agent. The declarative pipeline syntax  documentation says: 

            Inside a stage, the steps in the options directive are invoked before entering the agent or checking any when conditions.

             
            Is there any way to lock multiple stages for a single env.NODE_NAME?  Ideally, other subsequent stages could still run in parallel with other builds (i.e. so limiting executors to 1 or disabling concurrent builds are not options).
             

            rrueth Ryan Rueth added a comment - abayer , I'm trying to lock multiple stages based on the Jenkins node on which the stages will execute. It seems like the only way to do this is using the `options` block that you specified: stage( 'Parent' ) { options { lock( "${env.NODE_NAME}" ) } stages { stage( 'first child' ) { ... } stage( 'second child' ) { ... } } } Unfortunately, the resource this attempts to lock is `null`: Trying to acquire lock on [null]   It appears that this won't work because the stage's `options` directive occurs before entering the agent. The declarative pipeline syntax  documentation says:  Inside a  stage , the steps in the  options  directive are invoked before entering the  agent  or checking any  when  conditions.   Is there any way to lock multiple stages for a single env.NODE_NAME?  Ideally, other subsequent stages could still run in parallel with other builds (i.e. so limiting executors to 1 or disabling concurrent builds are not options).  
            nikolodus Dusan Nikolov added a comment -

            Just wanted to +1 on rrueth's comment regarding locking multiple stages with lock named after $NODE_NAME. Pretty much in the same situation, and I'd like to know if this can be supported - either by enabling options block to execute after agent has been assigned for a given stage, or through some other means if this is not an option.

            abayer do you know if this could be possible?

            nikolodus Dusan Nikolov added a comment - Just wanted to +1 on rrueth 's comment regarding locking multiple stages with lock named after $NODE_NAME. Pretty much in the same situation, and I'd like to know if this can be supported - either by enabling options block to execute after agent has been assigned for a given stage, or through some other means if this is not an option. abayer do you know if this could be possible?
            philip_abraham Philipp Abraham added a comment - - edited

            env is not available in options I think, but you could try getting the node name from the API:

            stage('Parent') {
               options {
                 lock("${currentBuild.getRawBuild().getExecutor().getOwner().getDisplayName()}")
               }
               stages {
                  stage('first child') {
                     ...
                  }
                  stage('second child') {
                     ...
                  }
               }
            }
            philip_abraham Philipp Abraham added a comment - - edited env is not available in options I think, but you could try getting the node name from the API: stage( 'Parent' ) { options { lock( "${currentBuild.getRawBuild().getExecutor().getOwner().getDisplayName()}" ) } stages { stage( 'first child' ) { ... } stage( 'second child' ) { ... } } }

            Hi, just wanted to add that we have the same problem where we need to lock multiple stages at once for atomicity (environment build and integration tests go together). Also, we need to parametrise the lock with the environment against which the build is done: it is ok to have parallel builds against different environments, but not against the same one.

            As it looks, currently the multistage lock is available only in the options section which does not support environment parameters. Hence, it looks that we can not achieve what we need.

            voz Andrii Vozniuk added a comment - Hi, just wanted to add that we have the same problem where we need to lock multiple stages at once for atomicity (environment build and integration tests go together). Also, we need to parametrise the lock with the environment against which the build is done: it is ok to have parallel builds against different environments, but not against the same one. As it looks, currently the multistage lock is available only in the options section which does not support environment parameters. Hence, it looks that we can not achieve what we need.
            bitwiseman Liam Newman added a comment -

            Bulk closing resolved issues.

            bitwiseman Liam Newman added a comment - Bulk closing resolved issues.

            People

              abayer Andrew Bayer
              rochdev Roch Devost
              Votes:
              19 Vote for this issue
              Watchers:
              40 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: