We've been having scheduling problems on our Jenkins server in the past few weeks. I haven't been able to figure out what is causing these problems, but I've noticed the following things:

1. Pipeline jobs are displayed as 'pending: waiting for executors on master', even though only 1 job is running, and there are 19 available executors.
2. Triggering a (non-pipeline) job manually appears to have no effect (no job appears in the queue), until several minutes later.
3. On both slaves and master, jobs are displayed as running, even though they have already finished minutes ago.(which can be seen by clicking on the job, where you can see that the job has been successful/failed).

I have not been able to pinpoint a reason for this. It doesn't appear to be related to load, as I've seen this issue with a single running (light-weight) job and 1 job in the queue, with low load and low disk access on the master.

This is making Jenkins (at the times when it occurs) unusable for us.