Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-23902

Matrix parent takes executor slot and blocks children

    XMLWordPrintable

Details

    Description

      Not sure when the behaviour changed, but the parent job used to take an extra "free" slot. Now it's taking an actual slot, which can end up being the only slot that one of its children can use, leading to an effective deadlock.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-23702 Matrix jobs consuming executors after upgrading to 1.570

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-13619 Add ability to throttle number of concurrent builds of matrix configurations per node

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            Ascending order - Click to sort in descending order
            danielbeck Daniel Beck added a comment -

            Did you set hudson.model.Hudson.flyweightSupport (see https://wiki.jenkins-ci.org/display/JENKINS/Features+controlled+by+system+properties )?

            danielbeck Daniel Beck added a comment - Did you set hudson.model.Hudson.flyweightSupport (see https://wiki.jenkins-ci.org/display/JENKINS/Features+controlled+by+system+properties )?
            jameshowe James Howe added a comment -

            No, but it should be defaulting to "true". Unless that's what changed and the wiki page is out of date?

            jameshowe James Howe added a comment - No, but it should be defaulting to "true". Unless that's what changed and the wiki page is out of date?
            danielbeck Daniel Beck added a comment -

            Are you using a plugin that changes how the queue or executor assignment work? (When in doubt, post your entire plugin list)

            Is this happening regularly or just in special circumstances? Can you reproduce the problem on a pristine Jenkins instance?

            danielbeck Daniel Beck added a comment - Are you using a plugin that changes how the queue or executor assignment work? (When in doubt, post your entire plugin list) Is this happening regularly or just in special circumstances? Can you reproduce the problem on a pristine Jenkins instance?
            jameshowe James Howe added a comment -

            Yeah, it could be a few plugins actually. I'll see what I can reproduce with.

            jameshowe James Howe added a comment - Yeah, it could be a few plugins actually. I'll see what I can reproduce with.
            oleg_nenashev Oleg Nenashev added a comment -

            Throttle Concurrent Builds plugin seems to be responsible.
            See https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/22

            oleg_nenashev Oleg Nenashev added a comment - Throttle Concurrent Builds plugin seems to be responsible. See https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/22
            oleg_nenashev Oleg Nenashev added a comment -

            Closing as Fixed.
            Please reopen if upgrading to throttle-concurrents-1.8.4 does not resolve the issue

            oleg_nenashev Oleg Nenashev added a comment - Closing as Fixed. Please reopen if upgrading to throttle-concurrents-1.8.4 does not resolve the issue
            jameshowe James Howe added a comment -

            Upgrading to throttle-concurrents-1.8.4 did fix it.

            However, something has broken it again.
            Now on Jenkins 1.590, throttle-concurrents-1.8.4, matrix-project-1.4, priority-sorter-2.9.
            I thought it was a 1.590->1.593 update, but downgrading didn't resolve it.

            jameshowe James Howe added a comment - Upgrading to throttle-concurrents-1.8.4 did fix it. However, something has broken it again. Now on Jenkins 1.590, throttle-concurrents-1.8.4, matrix-project-1.4, priority-sorter-2.9. I thought it was a 1.590->1.593 update, but downgrading didn't resolve it.
            jameshowe James Howe added a comment -

            Priority sorter is not in advanced mode.

            jameshowe James Howe added a comment - Priority sorter is not in advanced mode.
            jameshowe James Howe added a comment -

            Odd. With no apparent changes to config or plugins, they started using flyweight executors again.

            jameshowe James Howe added a comment - Odd. With no apparent changes to config or plugins, they started using flyweight executors again.
            simonrichter Simon Richter added a comment -

            I saw that happen when the slave node the job should have run on was down at that time and the matrix job was queued.

            simonrichter Simon Richter added a comment - I saw that happen when the slave node the job should have run on was down at that time and the matrix job was queued.
            danielbeck Daniel Beck added a comment -

            There have been fixes to the queue behavior until 1.610. That version or higher would be needed for any follow-up issue to make sense.

            danielbeck Daniel Beck added a comment - There have been fixes to the queue behavior until 1.610. That version or higher would be needed for any follow-up issue to make sense.

            People

              kohsuke Kohsuke Kawaguchi
              jameshowe James Howe
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: