Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19986

Throttle Concurrent Builds plugin doesn't appear to work

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • Jenkins 1.509.2, Throttle Concurrent Builds 1.8

      I just installed this today to try and avoid some issues I have with locks & latches. I have some multi-configuration jobs. I want to allow multiple jobs to run at the same time, but only 1 job of a given category to run per node. This sounds exactly like what the plugin was written for. So I went into my configuration and created a category and set Maximum Total Concurrent Builds to 0 and Maximum Concurrent Builds Per Node to 1. I then went to all of my jobs and assigned them to this category and checked the radio button for throttle as part of one or more categories.

      I then manually started two jobs from the same category to see if one waited for the other. It didn't work. Both jobs started on the same node. I also tried settings Maximum Concurrent Builds Per Node in the job itself to 1 and that didn't help either.

      What am I missing about this configuration?

          [JENKINS-19986] Throttle Concurrent Builds plugin doesn't appear to work

          Yves Hwang added a comment -

          I can confirm that builds were successful for us until early this week where we upgraded to Jenkins.
          Throttle 1.8 and Jenkins 1.536

          Yves Hwang added a comment - I can confirm that builds were successful for us until early this week where we upgraded to Jenkins. Throttle 1.8 and Jenkins 1.536

          Marcel Beister added a comment - - edited

          Maybe 20211 is the same problem.

          Marcel Beister added a comment - - edited Maybe 20211 is the same problem.

          This is still a problem in 1.538!

          Marcel Beister added a comment - This is still a problem in 1.538!

          I also had it working just fine until the latest jenkins update, and now with Throttle 1.8 and Jenkins 1.538 it seems like the plugin just doesn't work at all.

          It also seems broken with Throttle 1.7.2 and 1.7.1 on Jenkins 1.538.

          Michael Berman added a comment - I also had it working just fine until the latest jenkins update, and now with Throttle 1.8 and Jenkins 1.538 it seems like the plugin just doesn't work at all. It also seems broken with Throttle 1.7.2 and 1.7.1 on Jenkins 1.538.

          Not broken on 1.530.

          Michael Berman added a comment - Not broken on 1.530.

          Is this plugin still maintained or shall we look for something new? This bug was reported over a month ago and unfortunately nothing has happened yet. The plugin works find with Jenkins version 1.535 and is broken with all versions starting from 1.536. So I assume something in the api has changed which causes this plugin to fail...

          Marcel Beister added a comment - Is this plugin still maintained or shall we look for something new? This bug was reported over a month ago and unfortunately nothing has happened yet. The plugin works find with Jenkins version 1.535 and is broken with all versions starting from 1.536. So I assume something in the api has changed which causes this plugin to fail...

          Jon Hyman added a comment -

          Same here - it seems to have broken after updating from Jenkins 1.533 to 1.539. I tried updating to 1.540 to see if that fixes it but it is still broken.

          Jon Hyman added a comment - Same here - it seems to have broken after updating from Jenkins 1.533 to 1.539. I tried updating to 1.540 to see if that fixes it but it is still broken.

          at least I'm not the only one

          Marcel Beister added a comment - at least I'm not the only one

          Andrew Bayer added a comment -

          Sorry, I haven't looked at newer versions of Jenkins lately - I'll try to give this a look in the next week.

          Andrew Bayer added a comment - Sorry, I haven't looked at newer versions of Jenkins lately - I'll try to give this a look in the next week.

          That would be great! Thanks a lot...

          Marcel Beister added a comment - That would be great! Thanks a lot...

          jpschewe added a comment -

          Some more information that may be helpful fixing this plugin. I'm using the locks & latches plugin with a matrix job. It used to be the case that the sub-jobs that run on each node could run concurrently as long as they were on different nodes. Now it appears that they run in serial in most cases. I say most as I think I've seen them run concurrently recently as well, but in most case they are running in serial. I know this isn't the same plugin, but I suspect both plugins depend on similar APIs and something in there has changed.

          jpschewe added a comment - Some more information that may be helpful fixing this plugin. I'm using the locks & latches plugin with a matrix job. It used to be the case that the sub-jobs that run on each node could run concurrently as long as they were on different nodes. Now it appears that they run in serial in most cases. I say most as I think I've seen them run concurrently recently as well, but in most case they are running in serial. I know this isn't the same plugin, but I suspect both plugins depend on similar APIs and something in there has changed.

          I'm running into the same problem. Any idea on where the problem is coming from? Maybe I can help to debug / fix it? But I do need some pointers on where to look

          Phillip Gussow added a comment - I'm running into the same problem. Any idea on where the problem is coming from? Maybe I can help to debug / fix it? But I do need some pointers on where to look

          Linards L added a comment -

          Great. Another blocker for any upgrade from v1.529.

          Linards L added a comment - Great. Another blocker for any upgrade from v1.529.

          Andre Leitao added a comment - - edited

          I am currently migrating jenkins between machines, and i can confirm that in 1.537 with throttle 1.8 (old machine) everything is working fine, and with 1.538 with throttle 1.8 (new machine) it is broken, it doesn't work at all.

          (edit) Aparently the problem is with the labels, removing the labels did make everything start to work again (in the old machine i had restriction by node name, in the new i was trying to restrict by label). changing restriction to node name only in the new machine didn't work, i had to remove labels from the nodes as well.

          Other thing i found when restricting by label. I created a job with 10 downstream jobs all restricted by the same category (configured to restrict once across all nodes). When i run the job, it start working fine, with the "pending, one job already running" message. if i canceled the job running, then remaining jobs started running in all possible executors, and the message for the ones still in the queue (there were no more available executors) changed to "pending, 5 jobs already running"

          upgrading jenkins to 1.542, the problem persists with or without labels.

          Andre Leitao added a comment - - edited I am currently migrating jenkins between machines, and i can confirm that in 1.537 with throttle 1.8 (old machine) everything is working fine, and with 1.538 with throttle 1.8 (new machine) it is broken, it doesn't work at all. (edit) Aparently the problem is with the labels, removing the labels did make everything start to work again (in the old machine i had restriction by node name, in the new i was trying to restrict by label). changing restriction to node name only in the new machine didn't work, i had to remove labels from the nodes as well. Other thing i found when restricting by label. I created a job with 10 downstream jobs all restricted by the same category (configured to restrict once across all nodes). When i run the job, it start working fine, with the "pending, one job already running" message. if i canceled the job running, then remaining jobs started running in all possible executors, and the message for the ones still in the queue (there were no more available executors) changed to "pending, 5 jobs already running" upgrading jenkins to 1.542, the problem persists with or without labels.

          Stephen Rosen added a comment -

          We've tried configuring throttle concurrents on multiple builds ranging from 1.536 to 1.542, and it has never worked. I'm fairly sure we're experiencing this same error.
          I haven't carefully tracked all of the versions and attempted configurations, but we're running Ubuntu 12.04.3 servers on AWS with Jenkins as the only application that these servers are dedicated to. The jobs are free style jobs that pull in git repos and do a simple 'make clean && make test' shell action. Primarily, I've been trying to use the same configuration as the OP.

          This bug desperately needs resolution – we get spurious failures on a regular basis without throttle-concurrents.

          Stephen Rosen added a comment - We've tried configuring throttle concurrents on multiple builds ranging from 1.536 to 1.542, and it has never worked. I'm fairly sure we're experiencing this same error. I haven't carefully tracked all of the versions and attempted configurations, but we're running Ubuntu 12.04.3 servers on AWS with Jenkins as the only application that these servers are dedicated to. The jobs are free style jobs that pull in git repos and do a simple 'make clean && make test' shell action. Primarily, I've been trying to use the same configuration as the OP. This bug desperately needs resolution – we get spurious failures on a regular basis without throttle-concurrents.

          Jacek Migdal added a comment -

          Yet another user hit by this bug...

          Jacek Migdal added a comment - Yet another user hit by this bug...

          Andrew Bayer added a comment -

          Trying to look into this now.

          Andrew Bayer added a comment - Trying to look into this now.

          Andrew Bayer added a comment -

          Can one of you experiencing the problem give me a testcase I can use to reproduce the problem?

          Andrew Bayer added a comment - Can one of you experiencing the problem give me a testcase I can use to reproduce the problem?

          I have a Multi-Project Throttle Category set up with these settings:

          • Max Total: 0
          • Max Concurrent: 1

          I have two jobs with throttle concurrent builds turned on with these settings:

          • Throttle this project as part of one or more categories
          • Max Total: 0
          • Max Concurrent: 0
          • Multi-Project Throttle Category: 1 queue checked off

          On older Jenkins, one one of those jobs runs at a time on any node. When I upgrade Jenkins, it will happily run them both on two different executors of the same node.

          Michael Berman added a comment - I have a Multi-Project Throttle Category set up with these settings: Max Total: 0 Max Concurrent: 1 I have two jobs with throttle concurrent builds turned on with these settings: Throttle this project as part of one or more categories Max Total: 0 Max Concurrent: 0 Multi-Project Throttle Category: 1 queue checked off On older Jenkins, one one of those jobs runs at a time on any node. When I upgrade Jenkins, it will happily run them both on two different executors of the same node.

          Stephen Rosen added a comment -

          If you use AWS, launch with ami-bd6d40d4 in us-east and install Jenkins 1.537 and Throttle Concurrents with its dependencies. Otherwise, any server running Ubuntu 12.04 will probably do.

          Create a category with Maximum Total Concurrents = 1 and Maximum Concurrents Per Node = 1.
          Put two free style software jobs in this category, with run steps like "sleep 10000", and start them both.

          The erroneous behavior should present itself immediately.

          Stephen Rosen added a comment - If you use AWS, launch with ami-bd6d40d4 in us-east and install Jenkins 1.537 and Throttle Concurrents with its dependencies. Otherwise, any server running Ubuntu 12.04 will probably do. Create a category with Maximum Total Concurrents = 1 and Maximum Concurrents Per Node = 1. Put two free style software jobs in this category, with run steps like "sleep 10000", and start them both. The erroneous behavior should present itself immediately.

          Andrew Bayer added a comment -

          Ok, I legitimately can't reproduce this - it's throttling properly for me...

          Andrew Bayer added a comment - Ok, I legitimately can't reproduce this - it's throttling properly for me...

          Andrew Bayer added a comment -

          And that's testing with the latest on master and again with 1.8.0's release tag. Hrm.

          Andrew Bayer added a comment - And that's testing with the latest on master and again with 1.8.0's release tag. Hrm.

          Andrew Bayer added a comment -

          That was with Jenkins 1.536 - trying now with 1.538.

          Andrew Bayer added a comment - That was with Jenkins 1.536 - trying now with 1.538.

          Andrew Bayer added a comment -

          And still throttled fine with 1.538. Have you all been using the latest throttle plugin release (1.8.1) or just earlier?

          Andrew Bayer added a comment - And still throttled fine with 1.538. Have you all been using the latest throttle plugin release (1.8.1) or just earlier?

          I am at 1.7.1.

          Michael Berman added a comment - I am at 1.7.1.

          Andrew Bayer added a comment -

          That seemed to work too.

          Andrew Bayer added a comment - That seemed to work too.

          Stephen Rosen added a comment -

          Odd. I can confirm that I am no longer able to produce this error using the same versions and builds that experienced it before.
          My prior note has a typo, rather than "Maximum Total Concurrents = 1" we want "Maximum Total Concurrents = 0".

          However, I at least can no longer reproduce.

          Stephen Rosen added a comment - Odd. I can confirm that I am no longer able to produce this error using the same versions and builds that experienced it before. My prior note has a typo, rather than "Maximum Total Concurrents = 1" we want "Maximum Total Concurrents = 0". However, I at least can no longer reproduce.

          I can no longer reproduce the problem with Jenkins 1.544 and Plugin 1.8.1.

          Marcel Beister added a comment - I can no longer reproduce the problem with Jenkins 1.544 and Plugin 1.8.1.

          Martijn Laan added a comment - - edited

          Jenkins 1.544 + plugin 1.8.1 also fixes this problem for me

          Martijn Laan added a comment - - edited Jenkins 1.544 + plugin 1.8.1 also fixes this problem for me

          jpschewe added a comment -

          The plugin appears to work on a simple job.

          I just tested on a multi-configuration job and the throttle plugin doesn't appear to do anything. I'm using Jenkins 1.532.1 and throttle plugin 1.8.1. I created 1 category. I configured 2 multi-configuration jobs to be throttled by that category and to only allow 1 build per node. I then manually started both jobs and found that both jobs ran at the same time.

          jpschewe added a comment - The plugin appears to work on a simple job. I just tested on a multi-configuration job and the throttle plugin doesn't appear to do anything. I'm using Jenkins 1.532.1 and throttle plugin 1.8.1. I created 1 category. I configured 2 multi-configuration jobs to be throttled by that category and to only allow 1 build per node. I then manually started both jobs and found that both jobs ran at the same time.

          jpschewe added a comment -

          I just upgraded to Jenkins 1.532.2 and Throttle Concurrent Builds 1.8.1 and I'm still seeing 2 jobs from the same category build at the same time on the same node when there is a limit of 1 job per node for the specified category. In this case one job is from a matrix job and one is a free-style project job.

          jpschewe added a comment - I just upgraded to Jenkins 1.532.2 and Throttle Concurrent Builds 1.8.1 and I'm still seeing 2 jobs from the same category build at the same time on the same node when there is a limit of 1 job per node for the specified category. In this case one job is from a matrix job and one is a free-style project job.

          I can reproduce this with: Jenkins 1.551 and plugin 1.8.2

          A matrix with 2 Axis and (for testing) a little shell sleep is supposed to be run only 3 times per node.

          Configuration 1:
          Throttle this project alone *
          Maximum Total Concurrent Builds 0
          Maximum Concurrent Builds Per Node 3

          Result: All Executors on all Nodes are used

          Configuration 2:

          Global - Multi-Project Throttle Categories
          Category Name: kvm
          Maximum Total Concurrent Builds 0
          Maximum Concurrent Builds Per Node 3

          Throttle this project as part of one or more categories *
          Multi-Project Throttle Category *kvm

          Result: All Executors on all Nodes are used

          Can I help in debugging? where to look?

          Christian Prause added a comment - I can reproduce this with: Jenkins 1.551 and plugin 1.8.2 A matrix with 2 Axis and (for testing) a little shell sleep is supposed to be run only 3 times per node. Configuration 1: Throttle this project alone * Maximum Total Concurrent Builds 0 Maximum Concurrent Builds Per Node 3 Result: All Executors on all Nodes are used Configuration 2: Global - Multi-Project Throttle Categories Category Name: kvm Maximum Total Concurrent Builds 0 Maximum Concurrent Builds Per Node 3 Throttle this project as part of one or more categories * Multi-Project Throttle Category *kvm Result: All Executors on all Nodes are used Can I help in debugging? where to look?

          Looks broken on Jenkins 1.551 + 1.8.2 or Jenkins 1.557 + 1.8.2

          Jobs still stacking in queue instead concurrent running on different nodes.

          Igor Turchenko added a comment - Looks broken on Jenkins 1.551 + 1.8.2 or Jenkins 1.557 + 1.8.2 Jobs still stacking in queue instead concurrent running on different nodes.

          Sorry, I was wrong. I forgot to allow concurrent builds for job itself.

          Igor Turchenko added a comment - Sorry, I was wrong. I forgot to allow concurrent builds for job itself.

          I'm also unable to throttle even very simple matrix jobs, running Jenkins 1.565 and version 1.8.2 of the plugin. The job setup is:

          • Throttle this project alone
          • Maximum total concurrent builds: 0
          • Maximum concurrent builds per node: 1
          • One user-defined axis

          When I launch it no throttling is done – every available executor starts processing one of the configurations.

          I tried throttling by category (changing the project limits 0) and got the same behavior, both for generic limits and node-specific category limits.

          How can I help debug this?

          David Kaelbling added a comment - I'm also unable to throttle even very simple matrix jobs, running Jenkins 1.565 and version 1.8.2 of the plugin. The job setup is: Throttle this project alone Maximum total concurrent builds: 0 Maximum concurrent builds per node: 1 One user-defined axis When I launch it no throttling is done – every available executor starts processing one of the configurations. I tried throttling by category (changing the project limits 0) and got the same behavior, both for generic limits and node-specific category limits. How can I help debug this?

          Oleg Nenashev added a comment -

          See JENKINS-13619 for all issues related to MATRIX jobs.
          The new release of the plugin will provide throttling of matrix configurations in "Categories" mode at least

          Oleg Nenashev added a comment - See JENKINS-13619 for all issues related to MATRIX jobs. The new release of the plugin will provide throttling of matrix configurations in "Categories" mode at least

          Oleg Nenashev added a comment -

          Closing as a duplicate of JENKINS-13619 (fixed in throttle-concurrents-1.8.3)

          Oleg Nenashev added a comment - Closing as a duplicate of JENKINS-13619 (fixed in throttle-concurrents-1.8.3)

          Paul Gilmore added a comment -

          I think this issue has returned.
          I'm using Jenkins 2.32.1
          I have 10 executors on my Jenkins installation
          I set up 3 jobs, which all have these TCB options set:
          Throttle this project as part of one or more categories (set)
          Maximum Total Concurrent Builds = 1
          Maximum Concurrent Builds Per Node = 1
          List of parameters to check: Expensive

          If I queue up all 3 jobs, they all three run simultaneously. I expected them to block each other.

          Paul Gilmore added a comment - I think this issue has returned. I'm using Jenkins 2.32.1 I have 10 executors on my Jenkins installation I set up 3 jobs, which all have these TCB options set: Throttle this project as part of one or more categories (set) Maximum Total Concurrent Builds = 1 Maximum Concurrent Builds Per Node = 1 List of parameters to check: Expensive If I queue up all 3 jobs, they all three run simultaneously. I expected them to block each other.

            abayer Andrew Bayer
            jpschewe jpschewe
            Votes:
            8 Vote for this issue
            Watchers:
            24 Start watching this issue

              Created:
              Updated:
              Resolved: