Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44411

Support throttling of blocks without node() in the throttle() step

      We have a limit on the number of licenses we can use concurrently. Before pipeline as code we could use the throttle to limit the number of jobs to 16.
      With pipeline as code and 2.0 of the plugin we cannot limit jobs unless we create the jobs and trigger in parallel.
      Is there any intention to make the lateste plugin work in the same way as before without having to do parallel jobs

          [JENKINS-44411] Support throttling of blocks without node() in the throttle() step

          Robert Squire created issue -

          Oleg Nenashev added a comment -

          Hi Robert,

          I do not exactly understand what's wrong with the current implementation. throttle() step works well without Parallel, e.g. the following sample works well:

          throttle(['test_2']) {
            node() {
                  sh "sleep 500"
                  echo "Done"
            }
          }

          The thing is that the Pipeline actually starts, but node() step will synchronize on the category

          Oleg Nenashev added a comment - Hi Robert, I do not exactly understand what's wrong with the current implementation. throttle() step works well without Parallel, e.g. the following sample works well: throttle( ['test_2'] ) {   node() {         sh "sleep 500"         echo "Done"   } } The thing is that the Pipeline actually starts, but node() step will synchronize on the category

          Oleg Nenashev added a comment -

          Anyway, it worth providing such example in the docs

          Oleg Nenashev added a comment - Anyway, it worth providing such example in the docs

          Robert Squire added a comment -

          I have done the following:
          case (target_option != "HOST" && comp_tool == "ARM"):
          throttle(['ARM_LICENSES'])
          {
          stage("Build libraries")
          {
          try
          {
          sh ''' mkdir BUILD
          cd BUILD
          cmake -GNinja -DCTEST_GENERATE_XUNIT_FILES=ON -DCMAKE_TOOLCHAIN_FILE=$CBS/toolchains/ARMCortexR4Toolchain.cmake -DCBS_RUN_VPUML=OFF ${WORKSPACE}/sdmos/cmake/targets/repository
          ninja -v '''
          currentBuild.result = "${env.job_pass}"
          }
          catch (Exception err)
          {
          currentBuild.result = "${env.job_fail}"
          CommonUtilities.stage_failure("Build libraries")
          println err
          throw err
          }
          }
          }

          If it set ARM_LICENSES to 2 in Jenkins config then trigger threee pipelines that has this in I see all three jobs running the build stage at the same time.
          Do I need to move to beore the node as in the example?

          Robert Squire added a comment - I have done the following: case (target_option != "HOST" && comp_tool == "ARM"): throttle( ['ARM_LICENSES'] ) { stage("Build libraries") { try { sh ''' mkdir BUILD cd BUILD cmake -GNinja -DCTEST_GENERATE_XUNIT_FILES=ON -DCMAKE_TOOLCHAIN_FILE=$CBS/toolchains/ARMCortexR4Toolchain.cmake -DCBS_RUN_VPUML=OFF ${WORKSPACE}/sdmos/cmake/targets/repository ninja -v ''' currentBuild.result = "${env.job_pass}" } catch (Exception err) { currentBuild.result = "${env.job_fail}" CommonUtilities.stage_failure("Build libraries") println err throw err } } } If it set ARM_LICENSES to 2 in Jenkins config then trigger threee pipelines that has this in I see all three jobs running the build stage at the same time. Do I need to move to beore the node as in the example?

          Oleg Nenashev added a comment -

          Yes, throttling will work on the node() level only. so the node() definition should be within the throttle() block.

          abayer just to be flexible with support of other throttling scopes, maybe we could add something like a  ThrottlePlaceholderTask? Maybe a flyweight one... It would be similar with node() acquisition logic, which is not a state of the art design though. But it would help. WDYT?

          Oleg Nenashev added a comment - Yes, throttling will work on the node() level only. so the node() definition should be within the throttle() block. abayer just to be flexible with support of other throttling scopes, maybe we could add something like a  ThrottlePlaceholderTask? Maybe a flyweight one... It would be similar with node() acquisition logic, which is not a state of the art design though. But it would help. WDYT?

          Robert Squire added a comment -

          Ok if I put it before node then the other job waits.
          Thanks for the help

          Robert Squire added a comment - Ok if I put it before node then the other job waits. Thanks for the help

          Oleg Nenashev added a comment -

          If you do not mind, let's keep this bug open. I will discuss it with abayer and probably convert it to RFE

          Oleg Nenashev added a comment - If you do not mind, let's keep this bug open. I will discuss it with abayer and probably convert it to RFE

          Oleg Nenashev added a comment -

          Created https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/49 with documentation improvements. rob_squire I will appreciate feedback in the pull request.

          Regarding this issue, I am going to convert it to RFE

          Oleg Nenashev added a comment - Created https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/49 with documentation improvements. rob_squire I will appreciate feedback in the pull request. Regarding this issue, I am going to convert it to RFE
          Oleg Nenashev made changes -
          Summary Original: Throttle plugin 2.0 does not support single jobs New: Support throttling of blocks without node() in the throttle() step
          Oleg Nenashev made changes -
          Issue Type Original: Bug [ 1 ] New: New Feature [ 2 ]

            Unassigned Unassigned
            rob_squire Robert Squire
            Votes:
            12 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated: