I maintain a cross-platform open-source project which has to build many of its test cases in containers on a donated Linux virtual machine that is relatively weak in run-time resources (CPU/RAM), but it is capable of running QEMU and many containers or chroots with various-architecture environments preinstalled, and has sufficient storage to have those many chroots.

The machine is not provisioned sufficiently to even run as many idling agents as it has such environments, however when using them one at a time to build the project, it suffices.

So the idea was to allow Jenkins to access and use these environments agents, importantly with their unique sets of labels, etc. – but constrain that it should not run everything at once.

I had several ideas about approaching this over the past days, such as defining a concept of "sub-agents" accessible by ssh or chroot from a single instance of an ordinary agent.jar on that "real agent" (allowing any connection type to be used - SSH, Swarm, WebStart...), and exposed as Computer/Node items, and somehow fiddling with the run-time executor count (e.g. run up to 4 builds at any time using any of these nodes), or defining some new agent type (plugin?) without the benefit of solving the use-case for arbitrary agent types already existing in the ecosystem...

But so far the simplest solution that suited my case was to just extend the concept of delayed startup/idle shutdown of agents, and add support for "conflicting with" some other defined agent.

This does have downsides of lagging a minute (minimum idling time to stop the agent), or not spreading the loads evenly across chroots to allow some given maximum count of executors (considering how many stages are queued waiting for this chroot), maybe that feature could be bolted on later.

Still, so far just being able to pre-define those agents with metadata our pipeline needs for the test matrix, starting them one at a time, completing the queued stages, and handing over the processing resources to the next agent, is already something that brings benefit and only cost a few lines to implement (PR pending).