If you configure the master computer to have zero executors and save, the existing executors will continue to stick around, even though idle. You have to reboot (or just wait?) for them to go away.

When increasing the count, new executors are added immediately.