Details
-
Type:
Bug
-
Status: Open (View Workflow)
-
Priority:
Minor
-
Resolution: Unresolved
-
Component/s: throttle-concurrent-builds-plugin
-
Labels:
-
Similar Issues:
Description
See discussion at JENKINS-44747 and https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/50 - I'm legitimately not sure whether anyone will ever actually encounter it, but it's a theoretical possibility that throttled Pipeline threads could fail to actually throttle upon master restart due to the fix for JENKINS-44747.
Attachments
Issue Links
- relates to
-
-
- Resolved
-
Activity
I observed this issue recently after restarting one of my Jenkins masters that had Throttle Concurrents in use. Throttle Concurrents was supposed to keep the # of builds that use a specific external resource down to 30. After the restart, I verified that at least 40 builds were using that external resource, indicating that the throttle was not taking effect. I only observed this after the restart; before the restart, things were fine. The end result for me as a user was that the external resource was overwhelmed and all my jobs failed. It's disappointing to me that bugs like this have existed for over a year without resolution. It effectively means the Throttle Concurrents plugin is unusable, at least in my use case.
Basil Crow
added a comment - I observed this issue recently after restarting one of my Jenkins masters that had Throttle Concurrents in use. Throttle Concurrents was supposed to keep the # of builds that use a specific external resource down to 30. After the restart, I verified that at least 40 builds were using that external resource, indicating that the throttle was not taking effect. I only observed this after the restart; before the restart, things were fine. The end result for me as a user was that the external resource was overwhelmed and all my jobs failed. It's disappointing to me that bugs like this have existed for over a year without resolution. It effectively means the Throttle Concurrents plugin is unusable, at least in my use case. > It's disappointing to me that bugs like this have existed for over a year without resolution
Agreed, it is a disappointment. The plugin is open for adoption, and anybody is more than welcome to take an ownership or to propose a fix. Contributors move on sometimes, so plugins may stay in stale until somebody has interest to maintain them
It's not only during the master restart. It affects every build started before restart.
Due to some other issues I have to restart Jenkins every week (number of open files is rising to the point where Jenkins is not usable). As a result I'm observing this issue every week, and have to cancel builds started before reboot.
My observations are a little different than the issue description. For me, throttle for the running pipelines is forgotten during the reboot. All new Jobs after the reboot are throttling correctly but they are not respecting throttling that happened before reboot. This leads to builds deadlocks and is very frustrating because builds takes a really long time in my case.
I did not observed throttling problem during or after the restart but I haven't done much testing either. For me, it affects every single build started before reboot, making the Pipeline durability totally useless (meaning it can resume after master restart). I have to abort such build anyway because new builds will not see throttling of that build.
Do we have anyone fixing this bug ? This is really annoying if the Jenkins got restart, suddenly another set of builds got throttle, and max out our pool resource.
Wilson Ng
added a comment - Do we have anyone fixing this bug ? This is really annoying if the Jenkins got restart, suddenly another set of builds got throttle, and max out our pool resource.
No plan to work on it anytime soon. Pull requests will be appreciated, reviewed and integrated if possible.