Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13619

Add ability to throttle number of concurrent builds of matrix configurations per node

XMLWordPrintable

      It would be extremely useful to be able to throttle the number of concurrent builds of matrix configurations that can take place on a given node. The reason for this is when you have a matrix project in which all subprojects share the same parent workspace data. It is possible then to run concurrent builds across separate nodes, but not on the same node.

      The specific use case I am running into is for building Android. I have a matrix project that defines configurations for each device I wish to build. Every sub-project shares the same android source tree to build off of, so concurrent builds are not possible on the same node as they would clobber each other, but concurrent builds on separate nodes would be fine.

        1. throttle-concurrents-1.8.3-SNAPSHOT.hpi
          53 kB

          [JENKINS-13619] Add ability to throttle number of concurrent builds of matrix configurations per node

          Austen Dicken created issue -

          Louis Poudrier-Racette added a comment -

          We are using Jenkins to test code running on embedded devices.
          We have multiple nodes in our setup, each one has only one (1) embedded device attached to it.

          We have two types of Jobs:

          1. Some of our jobs require an embedded devices, they run tests on them. This type of jobs are matrix configurations because there are many versions of the embedded devices to test.
          2. Some of our jobs do not require the embedded devices, they only compile code, they don't run any tests.

          When tests are running on an embedded device (Job type 1), the node's CPU is mostly free. However, we cannot take advantage of this free CPU to run type 2 jobs because we can only have one executor per node.

          We cannot have more then one executor per node because we don't have the "ability to throttle number of concurrent builds of matrix configurations per node" for type 1 jobs.

          We need the "ability to throttle number of concurrent builds of matrix configurations per node" so that different configurations of a type 1 Job are not concurrently run on one node as each node only has one embedded device.

          This would improvement would greatly increase the effectiveness of our continuous integration setup.

          Louis Poudrier-Racette added a comment - We are using Jenkins to test code running on embedded devices. We have multiple nodes in our setup, each one has only one (1) embedded device attached to it. We have two types of Jobs: Some of our jobs require an embedded devices, they run tests on them. This type of jobs are matrix configurations because there are many versions of the embedded devices to test. Some of our jobs do not require the embedded devices, they only compile code, they don't run any tests. When tests are running on an embedded device (Job type 1), the node's CPU is mostly free. However, we cannot take advantage of this free CPU to run type 2 jobs because we can only have one executor per node. We cannot have more then one executor per node because we don't have the "ability to throttle number of concurrent builds of matrix configurations per node" for type 1 jobs. We need the "ability to throttle number of concurrent builds of matrix configurations per node" so that different configurations of a type 1 Job are not concurrently run on one node as each node only has one embedded device. This would improvement would greatly increase the effectiveness of our continuous integration setup.

          Adam PAPAI added a comment - - edited

          Do we have any improvements on this? We also need to throttle our matrix jobs.

          Is there something I can help?

          Adam PAPAI added a comment - - edited Do we have any improvements on this? We also need to throttle our matrix jobs. Is there something I can help?

          Kevin Neal added a comment -

          Would this work?
          https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

          Kevin Neal added a comment - Would this work? https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

          Chris Withers added a comment -

          It appears that the Throttle Concurrent Builds Plugin only works at the parent level, which isn't what is wanted here.

          Chris Withers added a comment - It appears that the Throttle Concurrent Builds Plugin only works at the parent level, which isn't what is wanted here.

          Chris Withers added a comment -

          I've raised an issue to fix this bug in the Throttle Concurrent Builds Plugin:

          https://issues.jenkins-ci.org/browse/JENKINS-16521

          Chris Withers added a comment - I've raised an issue to fix this bug in the Throttle Concurrent Builds Plugin: https://issues.jenkins-ci.org/browse/JENKINS-16521
          Oleg Nenashev made changes -
          Assignee Original: Andrew Bayer [ abayer ] New: Oleg Nenashev [ oleg_nenashev ]
          Oleg Nenashev made changes -
          Link New: This issue is duplicated by JENKINS-16521 [ JENKINS-16521 ]

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleJobProperty.java
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleMatrixProjectOptions.java
          src/main/resources/hudson/plugins/throttleconcurrents/Messages.properties
          src/main/resources/hudson/plugins/throttleconcurrents/ThrottleJobProperty/config.jelly
          src/main/resources/hudson/plugins/throttleconcurrents/ThrottleMatrixProjectOptions/config.jelly
          src/main/resources/hudson/plugins/throttleconcurrents/ThrottleMatrixProjectOptions/config.properties
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/c8280ad3ac3c4faf9b69e0401800813a5953dcfa
          Log:
          JENKINS-13619 - Added configuration options for Matrix project's throttling

          Signed-off-by: Oleg Nenashev <o.v.nenashev@gmail.com>

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/hudson/plugins/throttleconcurrents/ThrottleJobProperty.java src/main/java/hudson/plugins/throttleconcurrents/ThrottleMatrixProjectOptions.java src/main/resources/hudson/plugins/throttleconcurrents/Messages.properties src/main/resources/hudson/plugins/throttleconcurrents/ThrottleJobProperty/config.jelly src/main/resources/hudson/plugins/throttleconcurrents/ThrottleMatrixProjectOptions/config.jelly src/main/resources/hudson/plugins/throttleconcurrents/ThrottleMatrixProjectOptions/config.properties http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/c8280ad3ac3c4faf9b69e0401800813a5953dcfa Log: JENKINS-13619 - Added configuration options for Matrix project's throttling Signed-off-by: Oleg Nenashev <o.v.nenashev@gmail.com>
          Oleg Nenashev made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

            oleg_nenashev Oleg Nenashev
            cvpcs Austen Dicken
            Votes:
            7 Vote for this issue
            Watchers:
            22 Start watching this issue

              Created:
              Updated:
              Resolved: