Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-12480

Build step that runs multiple jobs in parallel, and blocks until all are complete

      We would like to kick off multiple test jobs as build steps, and block waiting for them all to finish. Currently the execute serially. But we have lots of available resources to run them, so it would be so nice to run them in parallel, and get the results in 1 hour, rather than many.

          [JENKINS-12480] Build step that runs multiple jobs in parallel, and blocks until all are complete

          Andrew Bayer added a comment -

          You can do this already - just specify a list of jobs to build in one step.

          Andrew Bayer added a comment - You can do this already - just specify a list of jobs to build in one step.

          I have this situation configured but it doesnt works fine, maybe this is a kind of incompatibility between plugins:
          I use "Parameterized Trigger" + "Run condition" + "Conditional build step" plugins to set a diamond execution but making the number of build in 2nd step variable (even 0) depending on the input parameters.

          -One problem is that with 0 value "Counter parameter factory" crash (Java exception) so I use conditional build step to check it. By now that is working as intended but it would be nice Counter parameter factory not to crash with 0 repetitions.

          -The main problem is when I launch another build it finish its main step but when it should continue to the 2nd step of the diamond it keeps waiting the downstreams builds of the other execution avoiding users to run more than 1 project at a time.

          Jacobo Jimenez added a comment - I have this situation configured but it doesnt works fine, maybe this is a kind of incompatibility between plugins: I use "Parameterized Trigger" + "Run condition" + "Conditional build step" plugins to set a diamond execution but making the number of build in 2nd step variable (even 0) depending on the input parameters. -One problem is that with 0 value "Counter parameter factory" crash (Java exception) so I use conditional build step to check it. By now that is working as intended but it would be nice Counter parameter factory not to crash with 0 repetitions. -The main problem is when I launch another build it finish its main step but when it should continue to the 2nd step of the diamond it keeps waiting the downstreams builds of the other execution avoiding users to run more than 1 project at a time.

          I've got the same troubles recently.
          We also use ""Parameterized Trigger" + "Run condition" + "Conditional build step"", but I tried to disable run-condition and conditional-buildstep, but the problem still persists.
          Jacob, did you try to disable plugins?

          Also I use 1.464 and I tried to downgrade to 1.460.

          Greg Temchenko added a comment - I've got the same troubles recently. We also use ""Parameterized Trigger" + "Run condition" + "Conditional build step"", but I tried to disable run-condition and conditional-buildstep, but the problem still persists. Jacob, did you try to disable plugins? Also I use 1.464 and I tried to downgrade to 1.460.

          Mishael Kim added a comment -

          I believe this is a bug in the current release (2.15 and 2.16) of the Parameterized Trigger Plugin. I've downgraded to version 2.10 and it is working again properly.

          The trick is to create only 1 build step, and then nest all of the triggers under that one build step so that they can all run in parallel.

          Mishael Kim added a comment - I believe this is a bug in the current release (2.15 and 2.16) of the Parameterized Trigger Plugin. I've downgraded to version 2.10 and it is working again properly. The trick is to create only 1 build step, and then nest all of the triggers under that one build step so that they can all run in parallel.

          Seeing this in 2.16. I'll try downgrading tomorrow and see if I can spot which version it first occurred in.

          Alexander Boyd added a comment - Seeing this in 2.16. I'll try downgrading tomorrow and see if I can spot which version it first occurred in.

          2.12 is the earliest version in which this works. It broke in 2.13 and started running all jobs serially, even if they're located in the same build step.

          Alexander Boyd added a comment - 2.12 is the earliest version in which this works . It broke in 2.13 and started running all jobs serially, even if they're located in the same build step.

          Hi everyone. I registered this issue in the "kickstarting" section on FreedomSponsors. This means that if you need this issue that bad, you can go to http://www.freedomsponsors.org/core/issue/80/build-step-that-runs-multiple-jobs-in-parallel-and-blocks-until-all-are-complete and offer a few bucks for it.

          Gardner Bickford added a comment - Hi everyone. I registered this issue in the "kickstarting" section on FreedomSponsors. This means that if you need this issue that bad, you can go to http://www.freedomsponsors.org/core/issue/80/build-step-that-runs-multiple-jobs-in-parallel-and-blocks-until-all-are-complete and offer a few bucks for it.

          Marko Macek added a comment - - edited

          Currently, one can run parallel jobs when specifying them in a single build step/"Projects to build". But this requires them to have the same parameters (except factory ones).

          But when specifying multiple "Projects to build" within a single build step, there's no option "Block until the triggered projects finish their builds" that would wait for all jobs of this build step.

          I'll be trying "File Generator Parameter Factory" now.... Doesn't work for me, because in many cases I need to run different jobs.

          Marko Macek added a comment - - edited Currently, one can run parallel jobs when specifying them in a single build step/"Projects to build". But this requires them to have the same parameters (except factory ones). But when specifying multiple "Projects to build" within a single build step, there's no option "Block until the triggered projects finish their builds" that would wait for all jobs of this build step. I'll be trying "File Generator Parameter Factory" now.... Doesn't work for me, because in many cases I need to run different jobs.

          ikedam added a comment -

          I get this ticket is a request of a new feature as described by Marco.
          If not, please let me know what the real problem is. and update the description.

          ikedam added a comment - I get this ticket is a request of a new feature as described by Marco. If not, please let me know what the real problem is. and update the description.

          I have the same issue as Marko. As a work-around, I'm using the Multijob Plugin [1] now, but that has other issues [2].

          [1] https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin
          [2] https://issues.jenkins-ci.org/browse/JENKINS-25636

          Sebastian Schuberth added a comment - I have the same issue as Marko. As a work-around, I'm using the Multijob Plugin [1] now, but that has other issues [2] . [1] https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin [2] https://issues.jenkins-ci.org/browse/JENKINS-25636

          Kunal W added a comment -

          We have to mention all the jobs in one step.
          Add all the jobs, separated by comma, in "Projects to build".
          Then check "Block until the triggered projects finish their builds"

          Works for me.

          Kunal W added a comment - We have to mention all the jobs in one step. Add all the jobs, separated by comma, in "Projects to build". Then check "Block until the triggered projects finish their builds" Works for me.

          Kunal W added a comment -

          Check the attached screenshot and configure the job.

          Kunal W added a comment - Check the attached screenshot and configure the job.

          Which version do you use to trigger multiple jobs in parallel ? We are using 2.35.2 and this is no possible. All jobs start in sequence.

          zdravko branzov added a comment - Which version do you use to trigger multiple jobs in parallel ? We are using 2.35.2 and this is no possible. All jobs start in sequence.

          in 2.35.2 it does not run jobs in parallel

          zdravko branzov added a comment - in 2.35.2  it does not run jobs in parallel

          Kunal W added a comment -

          2.32

          I have uploaded a screenshot of how I have configured it

          Kunal W added a comment - 2.32 I have uploaded a screenshot of how I have configured it

          If I have two different parameters to be passed for two different jobs, how can I achieve this?

          For example, I have two jobs to be triggered in parallel and both the jobs  have two different parameters to be passed, I have used "Predefined parameters", but I assume how jenkins will take the parameters for the two jobs?

          Anil Thatavarthi added a comment - If I have two different parameters to be passed for two different jobs, how can I achieve this? For example, I have two jobs to be triggered in parallel and both the jobs  have two different parameters to be passed, I have used "Predefined parameters", but I assume how jenkins will take the parameters for the two jobs?

          Please check if you have enough build processors active/free for multi-builds

          Andreas Welter added a comment - Please check if you have enough build processors active/free for multi-builds

            huybrechts huybrechts
            gorillacoder giuliano carlini
            Votes:
            20 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated: