Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35359

Limit concurrent builds for 1 MultiBranch project

      I am currently running the latest version of the Pipeline plugin and i have created a multi-branch job. Everything is running fine on my Jenkinsfile but i would like to limit only 1 concurrent build for my entire multi-branch project. I see that you can set the option individually for the sub-sequent (branch) job but i would like to know if there is a way to limit it for the entire folder.

          [JENKINS-35359] Limit concurrent builds for 1 MultiBranch project

          Jóhann Sigurðsson added a comment - - edited

          For anyone so inclined, I came up with a workaround by:

          1. Limiting Master node executors to Pipeline builds
          2. Creating a local slave agent and hooking it up to Jenkins to take care of other jobs

          Declarative script example:

          agent {
            label {
              label "Master"    
            }
          }
          

           

           

          Jóhann Sigurðsson added a comment - - edited For anyone so inclined, I came up with a workaround by: Limiting Master node executors to Pipeline builds Creating a local slave agent and hooking it up to Jenkins to take care of other jobs Declarative script example: agent { label { label "Master" } }    

          Is there a timeline for a fix for this issue – or a reasonable workaround?

          Peter Battaglia added a comment - Is there a timeline for a fix for this issue – or a reasonable workaround?

          Jesse Glick added a comment -

          Use lock (outside node!).

          Jesse Glick added a comment - Use lock (outside node !).

          I agree that the lock, even outside the context of node isn't really a solution here.

           

          I have two builds, A and B. Build A requires that it is build one at a time on a node. Build B does not. If what I've seen is correct, a resource locked by build A will prevent build B from running on that resource as well. This is not what I want.

          Am I missing something? There's no reason to waste the time waiting for A to finish when I have B builds waiting to go.

          Quentin Hartman added a comment - I agree that the lock, even outside the context of node isn't really a solution here.   I have two builds, A and B. Build A requires that it is build one at a time on a node. Build B does not. If what I've seen is correct, a resource locked by build A will prevent build B from running on that resource as well. This is not what I want. Am I missing something? There's no reason to waste the time waiting for A to finish when I have B builds waiting to go.

          Lockable resource do not solve this issue.  Let me clarify my use case:

          I have a shared Jenkins installation.  We have a number of teams using multi-branch pipeline jobs.  Sometimes a team will have a large number of branches build concurrently - for example, they have a lot of feature branches that all depend on a given snapshot, and are configured to build when an upstream project builds.  Another example is when a project is migrated from SVN to git.  Regardless of the actual trigger, the result is that every other project is effectively blocked out because we have no realistic way to limit the concurrent branches for a given project.  Implementing a lockable resource for every single project could technically solve the issue, but would be overly burdensome to administrate, and would then restrict us to a single concurrent branch, where we would prefer to allow up to four concurrent branches per project.

           

          Phillip Hayward added a comment - Lockable resource do not solve this issue.  Let me clarify my use case: I have a shared Jenkins installation.  We have a number of teams using multi-branch pipeline jobs.  Sometimes a team will have a large number of branches build concurrently - for example, they have a lot of feature branches that all depend on a given snapshot, and are configured to build when an upstream project builds.  Another example is when a project is migrated from SVN to git.  Regardless of the actual trigger, the result is that every other project is effectively blocked out because we have no realistic way to limit the concurrent branches for a given project.  Implementing a lockable resource for every single project could technically solve the issue, but would be overly burdensome to administrate, and would then restrict us to a single concurrent branch, where we would prefer to allow up to four concurrent branches per project.  

          Jesse Glick added a comment -

          By the way there is a Throttle builds branch property (Pipeline: rateLimitBuilds job property) which is appropriate for certain scenarios. Currently it only throttles builds within a branch project, though, not across branches. The feature is built into the multibranch system (i.e., branch-api, not this plugin) for historical reasons.

          Jesse Glick added a comment - By the way there is a Throttle builds branch property (Pipeline: rateLimitBuilds job property) which is appropriate for certain scenarios. Currently it only throttles builds within a branch project, though, not across branches. The feature is built into the multibranch system (i.e., branch-api , not this plugin) for historical reasons.

          I would like to request this too. I can only run 1 build at time (does not matter what branch).

          Using lockable resources would be a workaround - as I understand I need to configure jenkins manually (add lockable resource), the Jenkinsfile is not enough...

          Piotr Krukowiecki added a comment - I would like to request this too. I can only run 1 build at time (does not matter what branch). Using lockable resources would be a workaround - as I understand I need to configure jenkins manually (add lockable resource), the Jenkinsfile is not enough...

          Amit Dar added a comment - - edited

          I'm encountering this issue as well as a major drawback in the multi branch pipeline.

          the node i'm running on assigns the same workspace to two consecutive builds, which interrupt each other (the second build starts before the first is finished, this happens if two consecutive commits are done one after another during a short period of time)

          I kindly ask to address this issue -> would make a significant improvement!

          Amit Dar added a comment - - edited I'm encountering this issue as well as a major drawback in the multi branch pipeline. the node i'm running on assigns the same workspace to two consecutive builds, which interrupt each other (the second build starts before the first is finished, this happens if two consecutive commits are done one after another during a short period of time) I kindly ask to address this issue -> would make a significant improvement!

          Richard Lee added a comment -

          This seems like an anti-pattern. Your branch builds should be independent of one another. If they need to share some external resource that has limited capacity, there should be a 'lockable resource' for that thing.

          Richard Lee added a comment - This seems like an anti-pattern. Your branch builds should be independent of one another. If they need to share some external resource that has limited capacity, there should be a 'lockable resource' for that thing.

          Rich added a comment -

          Facing same issue with latest version of github-branch-source plugin (2.11.4). Is there a timeline when this can be fixed ? Is there a workaround other than locks ?

          Rich added a comment - Facing same issue with latest version of github-branch-source plugin (2.11.4). Is there a timeline when this can be fixed ? Is there a workaround other than locks ?

            Unassigned Unassigned
            zikphil Philippe Labat
            Votes:
            43 Vote for this issue
            Watchers:
            46 Start watching this issue

              Created:
              Updated: