Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7046

Block Build when downstream project is building

    • Icon: New Feature New Feature
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None
    • Dependant matrix builds across Linux and Solaris

      When a downstream build is in progress, Hudson should prevent the upstream builds from occurring until the downstream build is complete.

      For example:

      CompA->CompB->CompC->CompD

      If component C is building Hudson should block builds on A and B, until C has finished.

      It should then rebuild A, B and C before allowing D to proceed.

      As the current system stands, if a user presses "build now" on component A when component C is building, A builds immediately. This can be worked around in part by using the locks & latches plugin, but this has the side affect of disabling parallel Matrix builds which massively increases the build time (see JENKINS-2725). Further, it should NOT be necessary to use ANY plugins to provide this fundamental behaviour.

          [JENKINS-7046] Block Build when downstream project is building

          aggamemnon added a comment -

          "Maybe it's worth prototyping a setup where you zip up the output directory and archive that zip as an artifact, in case this issue is not resolved for some time.. though maybe it's very large and that's why you haven't tried that already?"

          Not feasible due to the large size of the build directories, the large number of them and the extra unnecessary complexity this would add to the build environment.

          aggamemnon added a comment - "Maybe it's worth prototyping a setup where you zip up the output directory and archive that zip as an artifact, in case this issue is not resolved for some time.. though maybe it's very large and that's why you haven't tried that already?" Not feasible due to the large size of the build directories, the large number of them and the extra unnecessary complexity this would add to the build environment.

          Alan Harder added a comment -

          Have you tried a setup with locks&latches plugin?

          Alan Harder added a comment - Have you tried a setup with locks&latches plugin?

          aggamemnon added a comment -

          Have you read the description?

          "This can be worked around in part by using the locks & latches plugin, but this has the side affect of disabling parallel Matrix builds which massively increases the build time (see JENKINS-2725). Further, it should NOT be necessary to use ANY plugins to provide this fundamental behaviour."

          aggamemnon added a comment - Have you read the description? "This can be worked around in part by using the locks & latches plugin, but this has the side affect of disabling parallel Matrix builds which massively increases the build time (see JENKINS-2725 ). Further, it should NOT be necessary to use ANY plugins to provide this fundamental behaviour."

          robsimon added a comment -

          On top the "lock" is released before the artifacts are copied back and the next job in line is kicked off. That means this approach does not work.

          robsimon added a comment - On top the "lock" is released before the artifacts are copied back and the next job in line is kicked off. That means this approach does not work.

          Alan Harder added a comment -

          yes, read it long ago..

          Alan Harder added a comment - yes, read it long ago..

          mleinart added a comment -

          Here's another use case I have:

          Large build with a large suite of tests (unit + 1.5hrs worth of selenium browser tests).

          Job A:
          Builds the app, starts it up
          Job B:
          Matrix job using a custom workspace that points to Job A's workspace
          4 elements in 1 Axis: 4 test suites that are run simultaneously

          We cant have Job A run while Job B is running because it'll wipe the workspace out. Job A has an SCM poll trigger which on a busy day will already have another run queued while B is still running. Rather than space SCM polls several hours apart, we'd rather the build for Job A be blocked by its downstream job.

          Locks and latches doesn't work because each instance of Job B (it's a matrix job, remember) tries to grab a lock, preventing the instances from executing concurrently.

          (Also note that we've found custom workspace to be the best option because the Copy Workspace has some crappy limitations (doesnt copy permissions, doesnt copy empty directories) and tarring/untarring a 1gb build is a waste of time and space)

          mleinart added a comment - Here's another use case I have: Large build with a large suite of tests (unit + 1.5hrs worth of selenium browser tests). Job A: Builds the app, starts it up Job B: Matrix job using a custom workspace that points to Job A's workspace 4 elements in 1 Axis: 4 test suites that are run simultaneously We cant have Job A run while Job B is running because it'll wipe the workspace out. Job A has an SCM poll trigger which on a busy day will already have another run queued while B is still running. Rather than space SCM polls several hours apart, we'd rather the build for Job A be blocked by its downstream job. Locks and latches doesn't work because each instance of Job B (it's a matrix job, remember) tries to grab a lock, preventing the instances from executing concurrently. (Also note that we've found custom workspace to be the best option because the Copy Workspace has some crappy limitations (doesnt copy permissions, doesnt copy empty directories) and tarring/untarring a 1gb build is a waste of time and space)

          mleinart added a comment -

          Here's a working implementation:
          https://github.com/mleinart/hudson/tree/JENKINS-7046

          It's basically copy/paste/substitute from the upstream option. A refactoring, especially of the display of the option (pulldown of Upstream, Downstream, Both?) may be desired.

          mleinart added a comment - Here's a working implementation: https://github.com/mleinart/hudson/tree/JENKINS-7046 It's basically copy/paste/substitute from the upstream option. A refactoring, especially of the display of the option (pulldown of Upstream, Downstream, Both?) may be desired.

          aggamemnon added a comment -

          Fantastic, any idea on when this will make it into a release?

          aggamemnon added a comment - Fantastic, any idea on when this will make it into a release?

          The change merged in 1.394. Sorry for the delay!

          Kohsuke Kawaguchi added a comment - The change merged in 1.394. Sorry for the delay!

          dogfood added a comment -

          Integrated in hudson_main_trunk #434
          [FIXED JENKINS-7046] - Add "Block build when downstream project is building" option
          recording the JENKINS-7046 RFE

          mleinart :
          Files :

          • core/src/main/resources/hudson/model/AbstractItem/configure-common.jelly
          • core/src/main/resources/hudson/model/Messages.properties
          • war/src/main/webapp/help/project-config/block-downstream-building.html
          • core/src/main/resources/lib/hudson/project/config-blockWhenDownstreamBuilding.jelly
          • core/src/main/java/hudson/model/AbstractProject.java

          Kohsuke Kawaguchi :
          Files :

          • changelog.html

          dogfood added a comment - Integrated in hudson_main_trunk #434 [FIXED JENKINS-7046] - Add "Block build when downstream project is building" option recording the JENKINS-7046 RFE mleinart : Files : core/src/main/resources/hudson/model/AbstractItem/configure-common.jelly core/src/main/resources/hudson/model/Messages.properties war/src/main/webapp/help/project-config/block-downstream-building.html core/src/main/resources/lib/hudson/project/config-blockWhenDownstreamBuilding.jelly core/src/main/java/hudson/model/AbstractProject.java Kohsuke Kawaguchi : Files : changelog.html

            Unassigned Unassigned
            aggamemnon aggamemnon
            Votes:
            21 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: