Details
-
Bug
-
Status: Resolved (View Workflow)
-
Major
-
Resolution: Fixed
-
None
-
amazon-ecs plugin version 1.16
-
-
v1.18
Description
If you have one agent template with labels defined, and you create a Pipeline job that requests nodes of the same label in a parallel block, only one ECS agent will be launched. As a result, the work is only done serially, rather than in parallel as expected, because only one executor is available. Example Pipeline code:
def tasks = [:]
tasks["task_1"] = {
stage ("task_1"){
node('mylabel') {
sh 'echo foo'
}
}
}
tasks["task_2"] = {
stage ("task_2"){
node('mylabel') {
sh 'echo bar'
}
}
}
parallel tasks
If you run two copies of this job, or any other job that uses the same label, an agent will be launched for each job that is running. And if the parallel block of your Pipeline job refers to two different labels, then two agents will be launched. The issue is only with uses of the same label within the parallel branches.
Attachments
Activity
Philipp Garbe
added a comment - The logic how agents are launched has changed in v1.18. It's still not completely in parallel, as there's a tradeoff between running completely in parallel and having started too many agents.
Jenkins calls `provision`, but even for parallel stages, it sets `excessWorkload` to 1. If the ECS task takes too long to start, `provision` is called again. This would lead to the situation that too many unnecessary agents are launched. That's why this code prevents further provisioning as long as another agent with the same label is in pending state.
Philipp Garbe
added a comment - The logic how agents are launched has changed in v1.18. It's still not completely in parallel, as there's a tradeoff between running completely in parallel and having started too many agents.
Jenkins calls `provision`, but even for parallel stages, it sets `excessWorkload` to 1. If the ECS task takes too long to start, `provision` is called again. This would lead to the situation that too many unnecessary agents are launched. That's why this code prevents further provisioning as long as another agent with the same label is in pending state.
https://github.com/jenkinsci/amazon-ecs-plugin/blob/master/src/main/java/com/cloudbees/jenkins/plugins/amazonecs/ECSCloud.java#L183-L198 
"And if the parallel block of your Pipeline job refers to two different labels"
Even if the different labels point to the same actual agent.