Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27968

Throttle concurrent builds and lockable resource do not work with block on upstream

      Setup

      Set Jenkins to have one node (master), and two executors.
      Create a throttle category my-category with global max 1.

      Create freestyle project A.
      Build step `sleep 10`.

      Create freestyle project B.
      Throttle on my-category.
      Build step `sleep 10`.

      Create freestyle project C.
      Throttle on my-category.
      Build step `sleep 10`.
      Post-build, trigger downstream project D.

      Create freestyle project D.
      Block on upstream jobs.
      Build step `sleep 10`.

      Trigger A, B, C, and D, in that order.

      Expected

      A starts.
      B starts.
      A finishes.
      B finishes.
      C starts.
      C finishes.
      D starts.
      D finishes.

      Actual

      A starts.
      B starts.
      A finishes.
      D starts.
      B finishes.
      C starts.
      D finishes.
      C finishes.

      D does not wait on C.

      I also tried lockable resource plugin with the same setup, but it didn't work either.

      I do not know of any workaround.

          [JENKINS-27968] Throttle concurrent builds and lockable resource do not work with block on upstream

          Oleg Nenashev added a comment -

          I would say there's no bug:
          1) You do not throttle D, hence Jenkins is eligible to run it when it has a free executor
          2) "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting
          3) C is blocked by throttling, hence Jenkins takes D

          Paul, do you agree?

          Oleg Nenashev added a comment - I would say there's no bug: 1) You do not throttle D, hence Jenkins is eligible to run it when it has a free executor 2) "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting 3) C is blocked by throttling, hence Jenkins takes D Paul, do you agree?

          Daniel Beck added a comment -

          There is no bug here. Jenkins schedules a build of D because nothing says it cannot. (It would probably prefer C as its queue item is older, but that is prevented.)

          Daniel Beck added a comment - There is no bug here. Jenkins schedules a build of D because nothing says it cannot. (It would probably prefer C as its queue item is older, but that is prevented.)

          Paul Draper added a comment - - edited

          > "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting.

          I'm not quite sure I know what you mean. "Block on upstream jobs" will certainly prevent a job from running if its upstream is in the queue.

          Create jobs A, B, C, where C is upstream to B. Have a single executor, and start A, B, and C. Even though B was queued before C, it will run after C, because the fact that C is queued prevents B from running.

          > Jenkins schedules a build of D because nothing says it cannot.

          "Block on upstream builds" says it cannot, as D's upstream C is still in the queue.

          "When this option is checked, Jenkins will prevent the project from building when a dependency of this project is in the queue, or building. The dependencies include the direct as well as the transitive dependencies."

          Paul Draper added a comment - - edited > "Block on upstream jobs" only prevents conflicts of RUNNING jobs, a task in the queue does not prevent D from starting. I'm not quite sure I know what you mean. "Block on upstream jobs" will certainly prevent a job from running if its upstream is in the queue. Create jobs A, B, C, where C is upstream to B. Have a single executor, and start A, B, and C. Even though B was queued before C, it will run after C, because the fact that C is queued prevents B from running. > Jenkins schedules a build of D because nothing says it cannot. "Block on upstream builds" says it cannot, as D's upstream C is still in the queue. "When this option is checked, Jenkins will prevent the project from building when a dependency of this project is in the queue, or building. The dependencies include the direct as well as the transitive dependencies."

          Daniel Beck added a comment -

          Would be interesting to know whether this has recently behaved as documented. I have a feeling this has been broken for a long time.

          Daniel Beck added a comment - Would be interesting to know whether this has recently behaved as documented. I have a feeling this has been broken for a long time.

          Paul Draper added a comment -

          @Daniel, I suspect you are correct. It's quite possible these have never worked together.

          It is present in latest (1.609), and in the latest stable (1.596.2).

          Paul Draper added a comment - @Daniel, I suspect you are correct. It's quite possible these have never worked together. It is present in latest (1.609), and in the latest stable (1.596.2).

          Oleg Nenashev added a comment -

          Javadoc documentation:
          > "True to keep builds of this project in queue when upstream projects are building. False by default to keep from breaking existing behavior."

          Jobs in the queue and pending jobs (assigned to executor, but not started) have Run.State.NOT_STARTED status (there's also Run.State.BUILDING status). Build::isBuilding() returns try for both statuses, so there's a risk of confusion.

          Paul, my bad. Your case is valid.

          The handling of the restrictions is based on Queue::getUnblockedItems(), so Jenkins ignores dependencies on blocked tasks like throttled runs.

          We just need to decide if it is a functional or a documentation bug. I'd vote fore the first option.

          Oleg Nenashev added a comment - Javadoc documentation: > "True to keep builds of this project in queue when upstream projects are building. False by default to keep from breaking existing behavior." Jobs in the queue and pending jobs (assigned to executor, but not started) have Run.State.NOT_STARTED status (there's also Run.State.BUILDING status). Build::isBuilding() returns try for both statuses, so there's a risk of confusion. Paul, my bad. Your case is valid. The handling of the restrictions is based on Queue::getUnblockedItems(), so Jenkins ignores dependencies on blocked tasks like throttled runs. We just need to decide if it is a functional or a documentation bug. I'd vote fore the first option.

          Oleg Nenashev added a comment -

          Unfortunately I was unable to find time to work on plugin over last year. I decided to step down as maintainer and to mark the plugin for adoption.

          If anybody is interested, let me know

          Oleg Nenashev added a comment - Unfortunately I was unable to find time to work on plugin over last year. I decided to step down as maintainer and to mark the plugin for adoption. If anybody is interested, let me know

            Unassigned Unassigned
            draperp Paul Draper
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: