Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49173

Maximum Concurrent Builds Per Node option not respected for Declarative Pipeline

    XMLWordPrintable

Details

    Description

      The attached screenshot shows a throttle category set up called "hw1" with a maximum concurrent builds of 8, and a maximum concurrent builds per node of 1

      When running with the following Pipeline syntax, everything works just fine:

      throttle(['hw1']) {
        node('swarm'){
          sleep 30
          }
      }

      with 4 nodes labeled 'swarm' this allows 1 job to be executed on each node, and lets us queue an infinite number of jobs that will wait for their turn, and execute with one on each node.

      With Declarative pipeline syntax:

      pipeline{
        agent {label 'swarm'} stages {
          stage ("have a nap"){
            steps {
              throttle(['hw1']) {
                sleep 30
                  }
               }
            }
         }
      }

      The behavior is that it will fire off 8 jobs, respecting Maximum Concurrent value that 'hw1' was set to, but not the maximum concurrent builds per node, so it fire off 8 jobs, stacking multiple on each node, and only allowing 1 additional job to be queued. Additional triggering of the job does not queue any additional job runs.

      Since we really want the entire job limited to 1 execution per node, I was looking around for a way to limit it. https://issues.jenkins-ci.org/browse/JENKINS-45140 describes the issue, and I attempted the solution listed there.

      properties([
        [
          $class: 'ThrottleJobProperty',
          categories: ['hw1'],
          limitOneJobWithMatchingParams: false,
          maxConcurrentPerNode: 0,
          maxConcurrentTotal: 0,
          paramsToUseForLimit: '',
          throttleEnabled: true,
          throttleOption: 'category'
        ],
      ])

      pipeline{
        agent {label 'swarm'}stages {
        stage ("have a nap"){
          steps {
           sleep 30
         }
          }
       }
      }

       

      Again the same behavior was observed.  8 Jobs fired, multiple jobs stacked per node, and only 1 additional job could be queued

      Attachments

        Activity

          abayer Andrew Bayer added a comment -

          Try throttle([‘hw1’]) in the options directive at the top level of the job? By the time you’re hitting the throttle step, you’re already on the agent.

          abayer Andrew Bayer added a comment - Try throttle( [‘hw1’] ) in the options directive at the top level of the job? By the time you’re hitting the throttle step, you’re already on the agent.
          tomlarrow Tom Larrow added a comment -

          Same behavior, will fire up 8 jobs, and put multiple on the same node.  Only allows 1 in the queue

          tomlarrow Tom Larrow added a comment - Same behavior, will fire up 8 jobs, and put multiple on the same node.  Only allows 1 in the queue
          madsopheim Mads Opheim added a comment -

          We experience the same behaviour in our project. Would really like this bug to be fixed, as this is what stops us from using (declarative) pipeline builds at the moment.

          madsopheim Mads Opheim added a comment - We experience the same behaviour in our project. Would really like this bug to be fixed, as this is what stops us from using (declarative) pipeline builds at the moment.
          initialzero Marc Benstein added a comment - - edited

          We would love to see this fixed. We kind of have a workaround using labels. But throttling with categories in the options section of a declarative pipeline would be best.

          initialzero Marc Benstein added a comment - - edited We would love to see this fixed. We kind of have a workaround using labels. But throttling with categories in the options section of a declarative pipeline would be best.
          pandorasbox963 pandoras box added a comment -

          tomlarrow - Have you found a work around for this ? I have a similar situation to address.

          pandorasbox963 pandoras box added a comment - tomlarrow - Have you found a work around for this ? I have a similar situation to address.
          unityer_zl zhang added a comment -

          Same behavior ,I have no methods to fix it~

          unityer_zl zhang added a comment - Same behavior ,I have no methods to fix it~

          People

            Unassigned Unassigned
            tomlarrow Tom Larrow
            Votes:
            9 Vote for this issue
            Watchers:
            17 Start watching this issue

            Dates

              Created:
              Updated: