EC2 plugin didn't noticed that Jenkins is in Shutdown mode and continued started instances (because jobs were in queue) and stopping instances (because jobs weren't started again) and starting, and stopping them again, and on, and on...

This caused $500 bill in Amazon over ~18 hours, because the instances were Windows (m5.xlarge), you pay for full hour and the timeout of inactive instances was set to 1 minute.

Implementation suggestion: If Jenkins is in shutdown mode, don't start new instances.