-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Minor
-
Component/s: ec2-plugin
-
Environment:Jenkins: 2.204.1
Amazon EC2 plugin (ID: ec2): 1.47
We have two templates for the ec2 plugin:
template1
label: test default
instance cap: 1
usage: NORMAL
template2 label: report instance cap: 3 usage: EXCLUSIVE
Jenkins doesn't create a new instance with template 2 when there are jobs in queue waiting for an available excutor on a running instance created with template 1, for example in our queue:
1. test_job (Restrict where this project can be run unchecked. Waiting for template1:test as all exectuors on template1:test are taken and we only want 1 instance with template1:test)
2. report_job (Restrict where this project can be run checked. Label expression: report. Waiting for template2:report as it does not exist)
report_job will be blocked until test_job is scheduled as Jenkins keeps trying to schedule test_job and ignores the request of an instance with template2 from report_job
UPDATE: I can confirm that this only happens when the first job in the queue has "Restrict where this project can be run" unchecked.
And I believe the problem is from Jenkins core not the ec2 plugin as the NodeProvisioner doesn't not look for all the labels as it should be (https://github.com/jenkinsci/jenkins/blob/jenkins-2.204.1/core/src/main/java/hudson/slaves/NodeProvisioner.java#L815)
Here is the log from hudson.slaves.NodeProvisioner class
Jan 16, 2020 9:38:08 PM FINER hudson.slaves.NodeProvisioner
Consulting hudson.plugins.ec2.NoDelayProvisionerStrategy@1cfb55fa provisioning strategy with state StrategyState{label=null, snapshot=LoadStatisticsSnapshot{definedExecutors=6, onlineExecutors=6, connectingExecutors=0, busyExecutors=6, idleExecutors=0, availableExecutors=0, queueLength=127}, plannedCapacitySnapshot=0, additionalPlannedCapacity=0}
Jan 16, 2020 9:38:09 PM FINER hudson.slaves.NodeProvisioner
Consulting hudson.slaves.NodeProvisioner$StandardStrategyImpl@6a13a6e5 provisioning strategy with state StrategyState{label=null, snapshot=LoadStatisticsSnapshot{definedExecutors=6, onlineExecutors=6, connectingExecutors=0, busyExecutors=6, idleExecutors=0, availableExecutors=0, queueLength=127}, plannedCapacitySnapshot=0, additionalPlannedCapacity=0}
Jan 16, 2020 9:38:09 PM FINE hudson.slaves.NodeProvisioner
Excess workload 117.81 detected. (planned capacity=0,connecting capacity=0,Qlen=117.81,available=0&0,online=6,m=0.106)
Jan 16, 2020 9:38:18 PM FINER hudson.slaves.NodeProvisioner
Consulting hudson.plugins.ec2.NoDelayProvisionerStrategy@1cfb55fa provisioning strategy with state StrategyState{label=null, snapshot=LoadStatisticsSnapshot{definedExecutors=6, onlineExecutors=6, connectingExecutors=0, busyExecutors=6, idleExecutors=0, availableExecutors=0, queueLength=127}, plannedCapacitySnapshot=0, additionalPlannedCapacity=0}
Jan 16, 2020 9:38:18 PM FINER hudson.slaves.NodeProvisioner
Consulting hudson.slaves.NodeProvisioner$StandardStrategyImpl@6a13a6e5 provisioning strategy with state StrategyState{label=null, snapshot=LoadStatisticsSnapshot{definedExecutors=6, onlineExecutors=6, connectingExecutors=0, busyExecutors=6, idleExecutors=0, availableExecutors=0, queueLength=127}, plannedCapacitySnapshot=0, additionalPlannedCapacity=0}
Jan 16, 2020 9:38:18 PM FINE hudson.slaves.NodeProvisioner
Excess workload 118.729 detected. (planned capacity=0,connecting capacity=0,Qlen=118.729,available=0&0,online=6,m=0.106)
