-
Improvement
-
Resolution: Unresolved
-
Minor
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline which has recently been submitted:
—
Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal
—
I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection as to which agent will receive the job at the moment it encounters the node() command in my Jenkinsfile.
The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname.
A better strategy would be one in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away.
[JENKINS-50306] "Still waiting to schedule task" indicates a flaw in the Jenkins pipelining design in my opinion
Jon B
created issue -
Jon B
made changes -
| Description |
Original:
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline: --- Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal --- I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection at the moment it encounters the node() command. The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname. A better strategy would be on in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away. |
New:
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline which has recently been submitted: — Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal — I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection at the moment it encounters the node() command. The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname. A better strategy would be on in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away. |
Jon B
made changes -
| Summary | Original: "Still waiting to schedule task" indicates a flaw in the design in my opinion | New: "Still waiting to schedule task" indicates a flaw in the Jenkins pipelining design in my opinion |
Jon B
made changes -
| Description |
Original:
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline which has recently been submitted: — Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal — I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection at the moment it encounters the node() command. The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname. A better strategy would be on in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away. |
New:
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline which has recently been submitted: — Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal — I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection as to which agent will receive the job at the moment it encounters the node() command in my Jenkinsfile. The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname. A better strategy would be on in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away. |
Jon B
made changes -
| Description |
Original:
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline which has recently been submitted: — Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal — I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection as to which agent will receive the job at the moment it encounters the node() command in my Jenkinsfile. The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname. A better strategy would be on in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away. |
New:
In my Jenkins server, I sometimes have a little bit of queueing and you'll see messages like this in a pipeline which has recently been submitted: — Still waiting to schedule task Waiting for next available executor on ip-172-31-141-11.us-west-2.compute.internal — I use node labels in order to select which agents may be used. However, the above message suggests to me that the Jenkins pipeline runner makes a selection as to which agent will receive the job at the moment it encounters the node() command in my Jenkinsfile. The reason that I believe this logic to be flawed is that the particular node in question (ip-172-31-141-11.us-west-2.compute.internal) might get killed off while the current job is running which suggests that my queued job will be stuck waiting forever because there is almost no chance that AWS will relaunch the same node with the same hostname. A better strategy would be one in which I request a node via something like node("mac") and then jenkins tells me that its waiting to schedule an executor on the next node labeled "mac" as opposed to selecting an individual machine which might go away. |
| Component/s | New: pipeline [ 21692 ] | |
| Component/s | Original: core [ 15593 ] |
| Component/s | New: workflow-durable-task-step-plugin [ 21715 ] | |
| Component/s | Original: pipeline [ 21692 ] |
| Component/s | New: throttle-concurrent-builds-plugin [ 15745 ] | |
| Component/s | Original: workflow-durable-task-step-plugin [ 21715 ] |
Jon B
made changes -
| Comment | [ I can confirm that yes I am using `throttle-concurrent-builds` plugin - a plugin which seems essential to helping not overwhelm the jenkins master and ensuring all of my pipeline users get a fair shot at the limited capacity. ] |
Jon B
made changes -
| Attachment | New: image-2021-01-05-11-28-11-800.png [ 53744 ] |