I see there was a change that was merged 5 years ago <https://github.com/jenkinsci/ec2-plugin/pull/199> - but it's still very confusing:
Determines how long slaves can remain idle before being stopped or terminated (see the Stop/Disconnect on Idle Timeout setting). Also determines how frequently nodes are checked for stopping/terminating.
Times are expressed in minutes, and a value of 0 (or an empty string) indicates that idle slaves should never be stopped/terminated. Positive values are simple time-outs after that many idle minutes. Negative values cause the node to time out when found idle within the specified number of minutes from the end of an hourly EC2 billing period.
As an example, let's say a node was started at 11:00.
If it has an idle timeout of 5 it will be checked every 5 minutes starting at 11:05. If it's checked at 11:30 and has been idle since 11:26 (for 4 minutes), it will be stopped or terminated.
If it has an idle timeout of -5 it will first be checked at 11:55, and then continuously until 12:00. If it becomes idle during that time it will be stopped or terminated.
Can someone explain why this setting isn't just a "don't let the instance sit around idle for N minutes" setting? This description is unnecessarily convoluted and confusing.
If I set an idle timeout of 20 minutes, and the last build for an instance was 30 minutes ago, then I expect that instance to have already been stopped, 10 minutes ago (plus/minus). Why doesn't that happen?
IF the reason for this is because instances are billed by the hour, and therefore an instance that lives for 5 minutes costs exactly as much as an instance that lived for 55 minutes, then can we please explain that? It would make so much more sense if it were explained that way.
And if that is the case, it also needs to be fixed for Spot instances, because Spot instances are not billed by the hour, but billed by the number of seconds it was used: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#billing-for-interrupted-spot-instances
So there is no benefit to snapping the termination check relative to the "billing hour", because that does not help at all with spot instances.