-
New Feature
-
Resolution: Unresolved
-
Blocker
When it is time to request a worker node, more than one can be requested.
It requests one, then rapidly checks for if its up (node needs a minute or so to come up when using spot instances), and fires another one, then another – until one of spots is finally there on time. This one becomes Jenkins slave, rest linger there until Spot Marketplace kills it.
Seems to be 1.42 specific: I reverted to version 1.41 and it seems to work fine.
This is how it looks in Jenkins log file:
(notice really short delays, only sir-bd78adpq gets into Jenkins slave list)
Jan 15, 2019 2:17:40 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Spot instance id in provision: sir-dvfib1pn
Jan 15, 2019 2:17:40 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate
Jan 15, 2019 2:17:40 PM hudson.plugins.ec2.EC2Cloud provision
INFO: We have now 1 computers, waiting for 1 more
Jan 15, 2019 2:17:40 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning current_ami (ami-XXXXX) from ec2-ec2 cloud with 1 executors. Remaining excess workload: 0
Jan 15, 2019 2:17:40 PM hudson.plugins.ec2.EC2Cloud$1 call
WARNING: SlaveTemplate{ami='ami-XXXXX', labels=''}
. Node terminated is neither pending, neither running, its
{2}. Terminate provisioningJan 15, 2019 2:17:48 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate{ami='ami-XXXXX', labels=''}. Attempting to provision slave needed by excess workload of 1 units
Jan 15, 2019 2:17:49 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Launching ami-XXXXX for template current_ami
Jan 15, 2019 2:17:50 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Spot instance id in provision: sir-bieg943m
Jan 15, 2019 2:17:50 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate{ami='ami-XXXXX', labels=''}. Attempting provision finished, excess workload: 0
Jan 15, 2019 2:17:50 PM hudson.plugins.ec2.EC2Cloud provision
INFO: We have now 1 computers, waiting for 1 more
Jan 15, 2019 2:17:50 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning current_ami (ami-XXXXX) from ec2-ec2 cloud with 1 executors. Remaining excess workload: 0
Jan 15, 2019 2:17:50 PM hudson.plugins.ec2.EC2Cloud$1 call
WARNING: SlaveTemplate{ami='ami-XXXXX', labels=''}. Node terminated is neither pending, neither running, its {2}
. Terminate provisioning
Jan 15, 2019 2:17:58 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate
Jan 15, 2019 2:17:59 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Launching ami-XXXXX for template current_ami
Jan 15, 2019 2:18:00 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Spot instance id in provision: sir-5mtr9erp
Jan 15, 2019 2:18:00 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate{ami='ami-XXXXX', labels=''}
. Attempting provision finished, excess workload: 0
Jan 15, 2019 2:18:00 PM hudson.plugins.ec2.EC2Cloud provision
INFO: We have now 1 computers, waiting for 1 more
Jan 15, 2019 2:18:00 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning current_ami (ami-XXXXX) from ec2-ec2 cloud with 1 executors. Remaining excess workload: 0
Jan 15, 2019 2:18:00 PM hudson.plugins.ec2.EC2Cloud$1 call
WARNING: SlaveTemplate
Jan 15, 2019 2:18:08 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate{ami='ami-XXXXX', labels=''}
. Attempting to provision slave needed by excess workload of 1 units
Jan 15, 2019 2:18:09 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Launching ami-XXXXX for template current_ami
Jan 15, 2019 2:18:10 PM hudson.plugins.ec2.SlaveTemplate provisionSpot
INFO: Spot instance id in provision: sir-bd78adpq
Jan 15, 2019 2:18:10 PM hudson.plugins.ec2.EC2Cloud provision
INFO: SlaveTemplate
. Attempting provision finished, excess workload: 0
Jan 15, 2019 2:18:10 PM hudson.plugins.ec2.EC2Cloud provision
INFO: We have now 1 computers, waiting for 1 more
Jan 15, 2019 2:18:10 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning current_ami (ami-XXXXX) from ec2-ec2 cloud with 1 executors. Remaining excess workload: 0
Jan 15, 2019 2:18:21 PM hudson.plugins.ec2.EC2Cloud$1 call