-
New Feature
-
Resolution: Unresolved
-
Minor
-
None
Amazon now provides termination notifications to their spot instances giving users a couple of minutes to react to a pending termination event. (https://aws.amazon.com/blogs/aws/new-ec2-spot-instance-termination-notices/)
It would be great if an EC2 spot instance slave could check for termination, and if so, have the option to automatically re queue any running jobs on the slave that is about to be terminated.
Currently we're using the Naginator plugin searching the output log for evidence that the slave was terminated to re-queue jobs. (which become new jobs entirely).
Under the current implementation of the plugin it would be the master iterating over each spot instances and looking for termination notices. Requeueing might be a bit complex as it will require Jenkins to provide a means to relaunch the same build job (which you might not want in some cases)