We recently upgraded our Jenkins instance to the latest LTS (2.138.2) and started noticing some irregularities with the EC2 plugin, namely:

• A lot more build nodes were created than before, for the same amount of incoming items in the build queue
• This caused an increased amount of AWS API calls, namely StopInstances event. Possibly related to the previous point.

This led us to try out the latest snapshot, 1.41-SNAPSHOT build on revision d4bdd6b83a7102330fd97ffbbd067edc34e47f97. A few hours later, our Jenkins instance had a deadlock problem that is described in the log I'm attaching.

Notice how the Gerrit plugin stops processing events after the EC2 plugin deadlock, this ultimately left our Jenkins unresponsive.

One important bit of information might be that we're using the Stop/Disconnect on Idle Timeout plugin option.

We're happy to provide more information if needed.