I am looking to use Throttle Concurrent Builds in a WorkFlow. I have jobs that perform automated, scheduled testing, as well as user-initiated jobs. All these jobs compete for the same set of hardware resources to run tests against. I'm currently using BuildFlows for all this and Throttle Concurrent Builds works fine. My problem is that I want to use the WorkFlow plugin for the automated jobs but TCB does not natively support WorkFlows.

      I've read that people dedicate a Node per unit of hardware to solve this problem. That's not a good solution for me since I can have 10-20 hardware units. I've also ready about people using a WorkFlow Stage, but this also doesn't work because of the user-initiated jobs that are outside of the WorkFlow.

      I've seen a presentation by CloudBees that appears to indicate that a WorkFlow can be used with any plugin, even if the plugin does not have native WorkFlow support. Is this possible for TCB and/or is there TCB support planned for WorkFlow?

      Thanks,

      Dave

          [JENKINS-31801] Pipeline Support in Throttle Concurrent Builds

          Dave Lawrence created issue -
          Andrew Bayer made changes -
          Labels Original: concurrent jenkins throttle workflow New: community-bee concurrent jenkins throttle workflow

          Andrew Bayer added a comment -

          Andrew Bayer added a comment - Looks like https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/32 is pertinent - diving in there.

          Andrew Bayer added a comment -

          So that change does make it possible to use TCB in its normal way with Workflow jobs - i.e., you can't use TCB within the Workflow itself, but you can use it to throttle groups of Workflow jobs, etc. I'm guessing that's not all you want, but is that a start?

          Andrew Bayer added a comment - So that change does make it possible to use TCB in its normal way with Workflow jobs - i.e., you can't use TCB within the Workflow itself, but you can use it to throttle groups of Workflow jobs, etc. I'm guessing that's not all you want, but is that a start?
          R. Tyler Croy made changes -
          Rank New: Ranked higher
          R. Tyler Croy made changes -
          Rank New: Ranked lower

          Andrew Bayer added a comment -

          Ping?

          Andrew Bayer added a comment - Ping?

          Oleg Nenashev added a comment - - edited

          daveml. Yes, this is a known case (was also a blocker at my previous company). Actually I would say that workflow's implementation is not very suitable to remote hardware management due to the failover robustness (which cannot be disabled IIRC). If Jenkins or its node restarts, WF will continue the execution and will restore ONLY it's own state. Currently the state of the remote stuff cannot be restored from Workflow => jobs may behave unreliably.

          > I've seen a presentation by CloudBees that appears to indicate that a WorkFlow can be used with any plugin

          I doubt it's correct. Could you refer this presentation? If it is misguiding, we will need to fix it.

          Oleg Nenashev added a comment - - edited daveml . Yes, this is a known case (was also a blocker at my previous company). Actually I would say that workflow's implementation is not very suitable to remote hardware management due to the failover robustness (which cannot be disabled IIRC). If Jenkins or its node restarts, WF will continue the execution and will restore ONLY it's own state. Currently the state of the remote stuff cannot be restored from Workflow => jobs may behave unreliably. > I've seen a presentation by CloudBees that appears to indicate that a WorkFlow can be used with any plugin I doubt it's correct. Could you refer this presentation? If it is misguiding, we will need to fix it.

          Andrew Bayer added a comment -

          jglick - pinging you on this to warn you I wanna talk this over with you tomorrow. =)

          Andrew Bayer added a comment - jglick - pinging you on this to warn you I wanna talk this over with you tomorrow. =)

          Dave Lawrence added a comment -

          Thanks to all who have responded. I appreciate the effort.

          To Andrew's point "...but you can use it to throttle groups of Workflow jobs..."
          Using TCB within the Workflow is where the value is. Applying a throttle over tan entire Workflow causes other problems as I'm sure you can imagine.

          Oleg:
          I'm sorry but I could not find the presentation. I know Cyrille gave it and it was older, perhaps Jan 2015. As for the Workflow failover, I understand the point, but it's not a concern for me. Hopefully TCB support can get worked into Workflow.

          Dave

          Dave Lawrence added a comment - Thanks to all who have responded. I appreciate the effort. To Andrew's point "...but you can use it to throttle groups of Workflow jobs..." Using TCB within the Workflow is where the value is. Applying a throttle over tan entire Workflow causes other problems as I'm sure you can imagine. Oleg: I'm sorry but I could not find the presentation. I know Cyrille gave it and it was older, perhaps Jan 2015. As for the Workflow failover, I understand the point, but it's not a concern for me. Hopefully TCB support can get worked into Workflow. Dave

            abayer Andrew Bayer
            daveml Dave Lawrence
            Votes:
            35 Vote for this issue
            Watchers:
            52 Start watching this issue

              Created:
              Updated:
              Resolved: