-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
-
Jenkins v2.12 / Ubuntu 14.04.4 x86_64 / Oracle Java 1.80_91-b14
-
Powered by SuggestiMate
Hi,
We've recently started seeing a problem auto-launching EC2 build slaves. The plugin detects that it needs to launch a slave to meet demand and that no current instances are runnign - it just doesn't launch anything.
Manual provisioning works.
Log (some info sanitised):
Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting to provision slave from template hudson.plugins.ec2.SlaveTemplate@6a756eca needed by excess workload of 1 units of label 'null' Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision WARNING: Label is null - can't calculate how many executors slave will have. Using 2 number of executors Considering launching ami-6c14310f for template Remote Ubuntu 14.04 Docker Machine Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Considering launching ami-6c14310f for template Remote Ubuntu 14.04 Docker Machine Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-6c14310f]}, {Name: subnet-id,Values: [subnet-XXXXX]}, {Name: instance.group-id,Values: [sg-XXXXX]}, {Name: key-name,Values: [XXXXX]}, {Name: instance-type,Values: [t2.small]}, {Name: tag:Name,Values: [Jenkins Build Slave]}],} No existing instance found - but cannot create new instance Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1
[JENKINS-36516] EC2 plugin: Plugin fails to automatically launch slaves to meet demand
Priority | Original: Minor [ 4 ] | New: Major [ 3 ] |
Description |
Original:
Hi, We've recently started seeing a problem auto-launching EC2 build slaves. The plugin detects that it needs to launch a slave to meet demand and that no current instances are runnign - it just doesn't launch anything. Log (some info sanitised): {noformat} Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting to provision slave from template hudson.plugins.ec2.SlaveTemplate@6a756eca needed by excess workload of 1 units of label 'null' Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision WARNING: Label is null - can't calculate how many executors slave will have. Using 2 number of executors Considering launching ami-6c14310f for template Remote Ubuntu 14.04 Docker Machine Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Considering launching ami-6c14310f for template Remote Ubuntu 14.04 Docker Machine Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-6c14310f]}, {Name: subnet-id,Values: [subnet-XXXXX]}, {Name: instance.group-id,Values: [sg-XXXXX]}, {Name: key-name,Values: [XXXXX]}, {Name: instance-type,Values: [t2.small]}, {Name: tag:Name,Values: [Jenkins Build Slave]}],} No existing instance found - but cannot create new instance Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 {noformat} |
New:
Hi, We've recently started seeing a problem auto-launching EC2 build slaves. The plugin detects that it needs to launch a slave to meet demand and that no current instances are runnign - it just doesn't launch anything. Manual provisioning works. Log (some info sanitised): {noformat} Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting to provision slave from template hudson.plugins.ec2.SlaveTemplate@6a756eca needed by excess workload of 1 units of label 'null' Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision WARNING: Label is null - can't calculate how many executors slave will have. Using 2 number of executors Considering launching ami-6c14310f for template Remote Ubuntu 14.04 Docker Machine Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Considering launching ami-6c14310f for template Remote Ubuntu 14.04 Docker Machine Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-6c14310f]}, {Name: subnet-id,Values: [subnet-XXXXX]}, {Name: instance.group-id,Values: [sg-XXXXX]}, {Name: key-name,Values: [XXXXX]}, {Name: instance-type,Values: [t2.small]}, {Name: tag:Name,Values: [Jenkins Build Slave]}],} No existing instance found - but cannot create new instance Jul 08, 2016 3:47:34 PM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 {noformat} |
Priority | Original: Major [ 3 ] | New: Blocker [ 1 ] |
I've observed the same issue in the latest Jenkins Version (2.10).
Manual provisioning works.
No existing instance found - but cannot create new instance Jul 13, 2016 6:05:33 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 6:05:41 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 6:05:42 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 6:05:42 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 6:05:43 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1
Can you try increasing the log level so we get the debug/fine level of logging?
Here you go Francis Upton
INFO: Finished EC2 alive slaves monitor. 402 ms Jul 13, 2016 1:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 1:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 229 ms Jul 13, 2016 1:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 1:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 184 ms Jul 13, 2016 2:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 2:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 162 ms Jul 13, 2016 2:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 2:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 200 ms Jul 13, 2016 2:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 2:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 175 ms Jul 13, 2016 2:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 2:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 234 ms Jul 13, 2016 2:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 2:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 182 ms Jul 13, 2016 2:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 2:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 185 ms Jul 13, 2016 3:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 3:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 155 ms Jul 13, 2016 3:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 3:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 138 ms Jul 13, 2016 3:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 3:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 163 ms Jul 13, 2016 3:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 3:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 167 ms Jul 13, 2016 3:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 3:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 166 ms Jul 13, 2016 3:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 3:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 169 ms Jul 13, 2016 4:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 4:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 165 ms Jul 13, 2016 4:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 4:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 161 ms Jul 13, 2016 4:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 4:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 163 ms Jul 13, 2016 4:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 4:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 167 ms Jul 13, 2016 4:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 4:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 163 ms Jul 13, 2016 4:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 4:55:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 157 ms Jul 13, 2016 5:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 5:05:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 172 ms Jul 13, 2016 5:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 5:15:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 171 ms Jul 13, 2016 5:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 5:25:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 175 ms Jul 13, 2016 5:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 5:35:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 160 ms Jul 13, 2016 5:39:14 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started Download metadata Jul 13, 2016 5:39:16 AM hudson.model.UpdateSite updateData INFO: Obtained the latest update center data file for UpdateSource default Jul 13, 2016 5:39:16 AM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller Jul 13, 2016 5:39:16 AM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.tasks.Ant.AntInstaller Jul 13, 2016 5:39:16 AM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.plugins.gradle.GradleInstaller Jul 13, 2016 5:39:16 AM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.tools.JDKInstaller Jul 13, 2016 5:39:16 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished Download metadata. 1,668 ms Jul 13, 2016 5:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started EC2 alive slaves monitor Jul 13, 2016 5:45:50 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished EC2 alive slaves monitor. 185 ms Jul 13, 2016 5:49:08 AM hudson.plugins.ec2.EC2OndemandSlave terminate INFO: Terminated EC2 instance (terminated): i-d81e5a5e Jul 13, 2016 5:49:08 AM hudson.plugins.ec2.EC2OndemandSlave terminate INFO: Removed EC2 instance from jenkins master: i-d81e5a5e Jul 13, 2016 5:52:24 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:52:26 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:52:26 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:52:26 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:52:34 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:52:36 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:52:36 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:52:36 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:52:44 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:52:46 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:52:46 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:52:46 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:52:54 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:52:56 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:52:56 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:52:56 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:53:04 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:53:06 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:53:06 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:53:06 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:53:14 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:53:16 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:53:16 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:53:16 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:53:24 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:53:25 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:53:25 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:53:25 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:53:34 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:53:35 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:53:35 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:53:35 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:53:44 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:53:45 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:53:45 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:53:45 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:53:54 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:53:55 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:53:55 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:53:56 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:54:02 AM jenkins.model.Jenkins$26 run INFO: Restart in 10 seconds Jul 13, 2016 5:54:12 AM jenkins.model.Jenkins$26 run SEVERE: Restarting VM as requested by anonymous Jul 13, 2016 5:54:12 AM jenkins.model.Jenkins cleanUp INFO: Stopping Jenkins Jul 13, 2016 5:54:12 AM jenkins.model.Jenkins$21 onAttained INFO: Started termination Jul 13, 2016 5:54:12 AM jenkins.model.Jenkins$21 onAttained INFO: Completed termination Jul 13, 2016 5:54:12 AM jenkins.model.Jenkins _cleanUpDisconnectComputers INFO: Starting node disconnection Jul 13, 2016 5:54:13 AM jenkins.model.Jenkins _cleanUpShutdownPluginManager INFO: Stopping plugin manager Jul 13, 2016 5:54:13 AM jenkins.model.Jenkins _cleanUpPersistQueue INFO: Persisting build queue Jul 13, 2016 5:54:13 AM jenkins.model.Jenkins _cleanUpAwaitDisconnects INFO: Waiting for node disconnection completion Jul 13, 2016 5:54:13 AM jenkins.model.Jenkins cleanUp INFO: Jenkins stopped Running from: /usr/share/jenkins/jenkins.war Jul 13, 2016 5:54:13 AM org.eclipse.jetty.util.log.JavaUtilLog info INFO: Logging initialized @380ms Jul 13, 2016 5:54:13 AM winstone.Logger logInternal INFO: Beginning extraction from war file Jul 13, 2016 5:54:13 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: Empty contextPath Jul 13, 2016 5:54:13 AM org.eclipse.jetty.util.log.JavaUtilLog info INFO: jetty-9.2.z-SNAPSHOT Jul 13, 2016 5:54:14 AM org.eclipse.jetty.util.log.JavaUtilLog info INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet Jenkins home directory: /var/lib/jenkins found at: EnvVars.masterEnvVars.get("JENKINS_HOME") Jul 13, 2016 5:54:15 AM org.eclipse.jetty.util.log.JavaUtilLog info INFO: Started w.@1e125710{/,file:/var/cache/jenkins/war/,AVAILABLE}{/var/cache/jenkins/war} Jul 13, 2016 5:54:15 AM org.eclipse.jetty.util.log.JavaUtilLog info INFO: Started ServerConnector@42eded9{HTTP/1.1}{0.0.0.0:8080} Jul 13, 2016 5:54:15 AM org.eclipse.jetty.util.log.JavaUtilLog info INFO: Started @1689ms Jul 13, 2016 5:54:15 AM winstone.Logger logInternal INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled Jul 13, 2016 5:54:15 AM jenkins.InitReactorRunner$1 onAttained INFO: Started initialization Jul 13, 2016 5:54:15 AM jenkins.InitReactorRunner$1 onAttained INFO: Listed all plugins Jul 13, 2016 5:54:15 AM jenkins.bouncycastle.api.SecurityProviderInitializer addSecurityProvider INFO: Initializing Bouncy Castle security provider. Jul 13, 2016 5:54:16 AM jenkins.bouncycastle.api.SecurityProviderInitializer addSecurityProvider INFO: Bouncy Castle security provider initialized. Jul 13, 2016 5:54:20 AM jenkins.InitReactorRunner$1 onAttained INFO: Prepared all plugins Jul 13, 2016 5:54:20 AM hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1 error INFO: Failed to instantiate optional component org.jenkinsci.plugins.workflow.steps.scm.GitStep$DescriptorImpl; skipping Jul 13, 2016 5:54:21 AM jenkins.InitReactorRunner$1 onAttained INFO: Started all plugins Jul 13, 2016 5:54:21 AM jenkins.InitReactorRunner$1 onAttained INFO: Augmented all extensions Jul 13, 2016 5:54:21 AM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect gscqascheduler113.fourvinternal.com Jul 13, 2016 5:54:21 AM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect gscqascheduler120.fourvinternal.com Jul 13, 2016 5:54:21 AM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect qa04scheduler.fourvinternal.com Jul 13, 2016 5:54:21 AM jenkins.InitReactorRunner$1 onAttained INFO: Loaded all jobs Jul 13, 2016 5:54:22 AM hudson.model.AsyncPeriodicWork$1 run INFO: Started Download metadata Jul 13, 2016 5:54:22 AM hudson.model.AsyncPeriodicWork$1 run INFO: Finished Download metadata. 1 ms Jul 13, 2016 5:54:22 AM org.jenkinsci.main.modules.sshd.SSHD start INFO: Started SSHD at port 37928 Jul 13, 2016 5:54:22 AM jenkins.InitReactorRunner$1 onAttained INFO: Completed initialization Jul 13, 2016 5:54:22 AM hudson.WebAppMain$3 run INFO: Jenkins is fully up and running [07/13/16 05:54:24] SSH Launch of gscqascheduler120.fourvinternal.com on 10.187.27.222 completed in 2,567 ms [07/13/16 05:54:26] SSH Launch of gscqascheduler113.fourvinternal.com on 10.187.29.0 completed in 4,673 ms [07/13/16 05:54:26] SSH Launch of qa04scheduler.fourvinternal.com on 10.187.29.100 completed in 4,840 ms Jul 13, 2016 5:56:01 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:56:04 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:56:04 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:56:04 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:56:11 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:56:12 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:56:12 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:56:12 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:56:21 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:56:22 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:56:22 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:56:22 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:56:31 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:56:32 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:56:32 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:56:32 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:56:41 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:56:42 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:56:42 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:56:43 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:56:51 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:56:52 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:56:52 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:56:52 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Jul 13, 2016 5:57:01 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:57:03 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:57:03 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],} No existing instance found - but cannot create new instance Jul 13, 2016 5:57:03 AM hudson.plugins.ec2.EC2Cloud provision INFO: Attempting provision - finished, excess workload: 1 Launching ami-e8e952ff for template GSC Cloud Slave Jul 13, 2016 5:57:26 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Launching ami-e8e952ff for template GSC Cloud Slave Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Jul 13, 2016 5:57:26 AM hudson.plugins.ec2.SlaveTemplate logProvisionInfo INFO: Setting Instance Initiated Shutdown Behavior : ShutdownBehavior.Terminate Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-e8e952ff]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-6aa49c1d]}, {Name: instance.group-id,Values: [sg-71d9bf16]}, {Name: key-name,Values: [VPC-GreySpark]}, {Name: instance-type,Values: [t2.medium]}, {Name: tag:ChargeLine,Values: [Internal-QA]}, {Name: tag:Lifecycle,Values: [Permanent]}, {Name: tag:Owner,Values: [rsamson@fourv.com"]}],}
Thanks for the additional tracing (it's good to get it from the beginning), but I also need the logging level changed so we get the "fine" tracing as well (not just "info")
Which version of the ec2 plugin are you using?