We have a docker swarm with multiple agents. We wanted to setup a few benchmark jobs to run on the same node, which happens to have an ssd so we labeled the node in the swarm as "ssd = true". Then we created an agent template which reserves enough resources so that two benchmark jobs would not work simultaneously on the same node. Afterwards we started two one after another (the same way as they would start in our pipeline). This started 2 agents - the first one started building and the second one was "offline".
The issue is that after the first job finished building - the second agent never gets online and the second job stays in the build queue forever. I couldn't find anything in the logs which suggests why that's happening. Not sure if this is a Docker Swarm Plugin bug or a Docker one?