I have two agents under a common label that I want to load balance using scoring by node loads. However, they are both on-demand agents. I have noticed that the load balancer will not consider a agent that is offline but can be brought online upon demand. My expectation was that the load balancer would bring an agent online if it was determined that it had a better score (offline agent would be completely unloaded). Perhaps this is a limitation in jenkins itself.
After a night of running, the first agent was taking multiple jobs while the second agent was offline the entire time. If I manually bring up the second agent, it will start taking jobs via the balancer.