• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None

      I'd like to be able to run matrix builds in parallel, currently that option doesn't appear in the configuration so I'm assuming it's not implemented yet.

          [JENKINS-6747] Support concurrent runs of matrix builds

          Axel Heider added a comment -

          I have several matrix jobs set with one configuration as touch-stone build. This works very well, first the parent job starts, then the touch-stone build and then all remaining jobs. They run in parallel - if there are enough free executors available. And different matrx job also run in parallel - again depending on how much executors are available.

          So I'm confused, what is the actual problem? Did you ever set up a sample configuration?

          Axel Heider added a comment - I have several matrix jobs set with one configuration as touch-stone build. This works very well, first the parent job starts, then the touch-stone build and then all remaining jobs. They run in parallel - if there are enough free executors available. And different matrx job also run in parallel - again depending on how much executors are available. So I'm confused, what is the actual problem? Did you ever set up a sample configuration?

          glundh added a comment -

          Axel: Have you tried running two concurrent builds of the same matrix project?

          Currently it is not possible. However FreeStyleProjects have had this support for quite a while, through the "Execute concurrent builds if necessary (beta)" option.

          I would also like to see the same functionality as the reporter of this issue. I hope I can break free enough time to start looking at a patch in the (probably quite distant) future.

          glundh added a comment - Axel: Have you tried running two concurrent builds of the same matrix project? Currently it is not possible. However FreeStyleProjects have had this support for quite a while, through the "Execute concurrent builds if necessary (beta)" option. I would also like to see the same functionality as the reporter of this issue. I hope I can break free enough time to start looking at a patch in the (probably quite distant) future.

          Sergey D added a comment -

          I found that concurrent run of the same matrix (multi-configuration) job is not possible.
          This is very disappointing and makes matrix job type completely unusable for us, since we loosing all advantage of parallel job execution.

          Sergey D added a comment - I found that concurrent run of the same matrix (multi-configuration) job is not possible. This is very disappointing and makes matrix job type completely unusable for us, since we loosing all advantage of parallel job execution.

          Sam Talebbeik added a comment -

          We have multi-configuration jobs and we can not run the jobs in parallel either. The first job starts and the second job automatically is put in the queue. Free slave servers are not a problem since we have enough free slave servers available to run the jobs in parallel. I even experimented with the "Throttle Concurrent Builds Plugin" and that does not seem to work with multi-configuration job. Having a check box or a plugin that allows a multi-configuration job to run in parallel will be a huge plus.

          Sam Talebbeik added a comment - We have multi-configuration jobs and we can not run the jobs in parallel either. The first job starts and the second job automatically is put in the queue. Free slave servers are not a problem since we have enough free slave servers available to run the jobs in parallel. I even experimented with the "Throttle Concurrent Builds Plugin" and that does not seem to work with multi-configuration job. Having a check box or a plugin that allows a multi-configuration job to run in parallel will be a huge plus.

          Marc Sanfacon added a comment -

          I actually was able to run several instances of a single matrix job concurrently. I manually modified the xml configuration file and set the flag concurrentBuild to true in the main matrix project as well as in all the projects configurations.

          I then executed 2 run of the matrix project and they were executed at the same time.

          Two things I noticed:
          1- The concurrentBuild flag is resetted when I modify the configuration from the Jenkins UI (probably normal)
          2- The workspace files are created under the main project name (missing the @2 for example). Basically means that if my project is "Test" and the matrix is Debug & Release, when more than one instance of the job starts, the main project files will be located in workspace/Test & workspace/Test@2, ... But the children job will always be under workspace/Test/, meaning workspace/Test/Debug & workspace/Test/Debug@2, etc.

          I don't see any reason why not to fix both issues and have that in Jenkins.

          Comments?

          Marc Sanfacon added a comment - I actually was able to run several instances of a single matrix job concurrently. I manually modified the xml configuration file and set the flag concurrentBuild to true in the main matrix project as well as in all the projects configurations. I then executed 2 run of the matrix project and they were executed at the same time. Two things I noticed: 1- The concurrentBuild flag is resetted when I modify the configuration from the Jenkins UI (probably normal) 2- The workspace files are created under the main project name (missing the @2 for example). Basically means that if my project is "Test" and the matrix is Debug & Release, when more than one instance of the job starts, the main project files will be located in workspace/Test & workspace/Test@2, ... But the children job will always be under workspace/Test/, meaning workspace/Test/Debug & workspace/Test/Debug@2, etc. I don't see any reason why not to fix both issues and have that in Jenkins. Comments?

          crusius added a comment -

          A major deal for our installation too: we have to do continuous integration builds on various platforms and have the hardware ready to make that happen, but can't do it due to this issue.

          crusius added a comment - A major deal for our installation too: we have to do continuous integration builds on various platforms and have the hardware ready to make that happen, but can't do it due to this issue.

          Indeed - the problem is not fixed at all. Running a multi-configuration task having 1 user defined axis with 2 values (e.g. "job1" and "job2") executing a permanent-ping to localhost will block the same, again started task. There are enough build slots available.

          For us it will save much time, because our builds currently starting two processes and each uses only a single CPU's on a I7-machine and we need to build two of them. That means - we have enough system resources (CPU-time and RAM) to start the second task parallel...

          Please, can you have a look at the problem?

          Thx

          Sven Appenrodt added a comment - Indeed - the problem is not fixed at all. Running a multi-configuration task having 1 user defined axis with 2 values (e.g. "job1" and "job2") executing a permanent-ping to localhost will block the same, again started task. There are enough build slots available. For us it will save much time, because our builds currently starting two processes and each uses only a single CPU's on a I7-machine and we need to build two of them. That means - we have enough system resources (CPU-time and RAM) to start the second task parallel... Please, can you have a look at the problem? Thx

          Andrew Melo added a comment -

          I'm in the same boat as the others. We actually have access to a giant VM farm where we can spin up (nearly) unbounded numbers of VMs, but the matrix builds don't take advantage of that. I'm looking to see if I can't at least make the flyweight build run concurrently.

          Andrew Melo added a comment - I'm in the same boat as the others. We actually have access to a giant VM farm where we can spin up (nearly) unbounded numbers of VMs, but the matrix builds don't take advantage of that. I'm looking to see if I can't at least make the flyweight build run concurrently.

          Lena F added a comment -

          Me too. Most of our tests are matrix, we can compile in parallel but the tests are just run one after the other. Any news on fixing this?

          Lena F added a comment - Me too. Most of our tests are matrix, we can compile in parallel but the tests are just run one after the other. Any news on fixing this?

          It will be really great feature to have matrix concurrent builds.

          Natalia Naumova added a comment - It will be really great feature to have matrix concurrent builds.

          Adam Spektor added a comment -

          We have a lot of Jobs and more then 60% of them are matrix so actually its really blocker for us to boost up our job execution time
          Thanks.

          Adam Spektor added a comment - We have a lot of Jobs and more then 60% of them are matrix so actually its really blocker for us to boost up our job execution time Thanks.

          OK - seems, this problem ist not popular so i tested several changes in the configuration. Seems it is a display-problem only. If you browse to the job-configuration, open all config.xml files (also in all!! sub folders) and change the entry: <concurrentBuild>false</concurrentBuild> to 'true' then you can use parallel builds with this kind of job, after reloading the configuration.

          Maybe using this knowlege, this bug can be handled, so the configurations can be saved in a official way...

          Sven Appenrodt added a comment - OK - seems, this problem ist not popular so i tested several changes in the configuration. Seems it is a display-problem only. If you browse to the job-configuration, open all config.xml files (also in all!! sub folders) and change the entry: <concurrentBuild>false</concurrentBuild> to 'true' then you can use parallel builds with this kind of job, after reloading the configuration. Maybe using this knowlege, this bug can be handled, so the configurations can be saved in a official way...

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          http://jenkins-ci.org/commit/jenkins/c7d8b541787f36b50f7f0aaf91fc334a233f405b
          Log:
          [FIXED JENKINS-6747] support concurrent runs of matrix builds.

          The actual fix was in c86aa67953ca0d7d5e8a0ede276a659cdbe8ba6b

          Compare: https://github.com/jenkinsci/jenkins/compare/25b6943...c7d8b54

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html http://jenkins-ci.org/commit/jenkins/c7d8b541787f36b50f7f0aaf91fc334a233f405b Log: [FIXED JENKINS-6747] support concurrent runs of matrix builds. The actual fix was in c86aa67953ca0d7d5e8a0ede276a659cdbe8ba6b Compare: https://github.com/jenkinsci/jenkins/compare/25b6943...c7d8b54

          dogfood added a comment -

          Integrated in jenkins_ui-changes_branch #30
          [FIXED JENKINS-6747] support concurrent runs of matrix builds. (Revision c7d8b541787f36b50f7f0aaf91fc334a233f405b)

          Result = SUCCESS
          Kohsuke Kawaguchi : c7d8b541787f36b50f7f0aaf91fc334a233f405b
          Files :

          • changelog.html

          dogfood added a comment - Integrated in jenkins_ui-changes_branch #30 [FIXED JENKINS-6747] support concurrent runs of matrix builds. (Revision c7d8b541787f36b50f7f0aaf91fc334a233f405b) Result = SUCCESS Kohsuke Kawaguchi : c7d8b541787f36b50f7f0aaf91fc334a233f405b Files : changelog.html

            kohsuke Kohsuke Kawaguchi
            esmalling Eric Smalling
            Votes:
            19 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: