-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
-
Jenkins LTS 2.60.3
Plugins:
Name Version Enabled
ace-editor 1.1 true
authentication-tokens 1.3 true
bouncycastle-api 2.16.2 true
branch-api 2.0.11 true
cloudbees-folder 6.1.2 true
credentials 2.1.14 true
credentials-binding 1.13 true
display-url-api 2.0 true
docker-commons 1.8 true
docker-workflow 1.12 true
durable-task 1.14 true
git 3.5.1 true
git-client 2.5.0 true
git-server 1.7 true
handlebars 1.1.1 true
icon-shim 2.0.3 true
jackson2-api 2.7.3 true
jquery-detached 1.2.1 true
junit 1.21 true
mailer 1.20 true
matrix-project 1.11 true
momentjs 1.1.1 true
pipeline-build-step 2.5.1 true
pipeline-graph-analysis 1.5 true
pipeline-input-step 2.8 true
pipeline-milestone-step 1.3.1 true
pipeline-model-api 1.1.9 true
pipeline-model-declarative-agent 1.1.1 true
pipeline-model-definition 1.1.9 true
pipeline-model-extensions 1.1.9 true
pipeline-rest-api 2.9 true
pipeline-stage-step 2.2 true
pipeline-stage-tags-metadata 1.1.9 true
pipeline-stage-view 2.9 true
plain-credentials 1.4 true
scm-api 2.2.1 true
script-security 1.33 true
ssh-credentials 1.13 true
structs 1.10 true
workflow-aggregator 2.5 true
workflow-api 2.20 true
workflow-basic-steps 2.6 true
workflow-cps 2.39 true
workflow-cps-global-lib 2.8 true
workflow-durable-task-step 2.15 true
workflow-job 2.12.2 true
workflow-multibranch 2.16 true
workflow-scm-step 2.6 true
workflow-step-api 2.12 true
workflow-support 2.14 trueJenkins LTS 2.60.3 Plugins: Name Version Enabled ace-editor 1.1 true authentication-tokens 1.3 true bouncycastle-api 2.16.2 true branch-api 2.0.11 true cloudbees-folder 6.1.2 true credentials 2.1.14 true credentials-binding 1.13 true display-url-api 2.0 true docker-commons 1.8 true docker-workflow 1.12 true durable-task 1.14 true git 3.5.1 true git-client 2.5.0 true git-server 1.7 true handlebars 1.1.1 true icon-shim 2.0.3 true jackson2-api 2.7.3 true jquery-detached 1.2.1 true junit 1.21 true mailer 1.20 true matrix-project 1.11 true momentjs 1.1.1 true pipeline-build-step 2.5.1 true pipeline-graph-analysis 1.5 true pipeline-input-step 2.8 true pipeline-milestone-step 1.3.1 true pipeline-model-api 1.1.9 true pipeline-model-declarative-agent 1.1.1 true pipeline-model-definition 1.1.9 true pipeline-model-extensions 1.1.9 true pipeline-rest-api 2.9 true pipeline-stage-step 2.2 true pipeline-stage-tags-metadata 1.1.9 true pipeline-stage-view 2.9 true plain-credentials 1.4 true scm-api 2.2.1 true script-security 1.33 true ssh-credentials 1.13 true structs 1.10 true workflow-aggregator 2.5 true workflow-api 2.20 true workflow-basic-steps 2.6 true workflow-cps 2.39 true workflow-cps-global-lib 2.8 true workflow-durable-task-step 2.15 true workflow-job 2.12.2 true workflow-multibranch 2.16 true workflow-scm-step 2.6 true workflow-step-api 2.12 true workflow-support 2.14 true
The issue occours if you have a "Multibranch Pipeline" job that takes some time such as:
pipeline {
agent any
stages {
stage('only'){
steps {
checkout scm
sh 'sleep 300'
}
}
}
}
This gets automatically detected by the pipeline and executed. This can also be kicked off manually by going into the job and clicking the run button beside the branch (e.g. 'master'). If this button is pushed twice in succession (or two branches are committed at the same time) two instances of the job will run. If there is a single build executor that these jobs can run on (I have only seen this when the executor is separate from the "master" node), the first will start running. The second will do some pipeline action to identify what needs to be run, but then wait in the queue for the first job to complete. Once the first job has completed the second job will run to completion. For the above job, the first job will show as taking the appropriate five minutes. However, the second job will show as taking ten minutes....it is including the time it was waiting for an executor! If this is done with many jobs, the time will include all the time that each job had to wait in the queue. When looking at the historical builds, it will look like it took ten minutes to execute even though it was five of waiting and five of executing. This will also affect the projected build time for the next job run.
This also shows "10 minutes building on an executor" if the metrics plugin is installed.
The solution to this is to not include the time spent waiting for an executor in the build time recorded. I believe the matrix jobs had a similar issue that was fixed in Issue #8112.
[JENKINS-46569] Job execution time includes waiting time
Thomas Kent
created issue -
Daniel Daehler
made changes -
| Issue Type | Original: Improvement [ 4 ] | New: Bug [ 1 ] |
Thomas de Grenier de Latour
made changes -
| Description |
Original:
The issue occours if you have a "Multibranch Pipeline" job that takes some time such as: {code:java} pipeline { agent any stages { stage('only'){ steps { checkout scm sh 'sleep 300' } } } }{code} This gets automatically detected by the pipeline and executed. This can also be kicked off manually by going into the job and clicking the run button beside the branch (e.g. 'master'). If this button is pushed twice in succession (or two branches are committed at the same time) two instances of the job will run. If there is a single build executor that these jobs can run on (I have only seen this when the executor is separate from the "master" node), the first will start running. The second will do some pipeline action to identify what needs to be run, but then wait in the queue for the first job to complete. Once the first job has completed the second job will run to completion. For the above job, the first job will show as taking the appropriate five minutes. However, the second job will show as taking ten minutes....it is including the time it was waiting for an executor! If this is done with many jobs, the time will include all the time that each job had to wait in the queue. When looking at the historical builds, it will look like it took ten minutes to execute even though it was five of waiting and five of executing. This will also affect the projected build time for the next job run. This also shows "10 minutes building on an executor" if the metrics plugin is installed. The solution to this is to not include the time spent waiting for an executor in the build time recorded. I believe the matrix jobs had a similar issue that was fixed in Issue #8112. |
New:
The issue occours if you have a "Multibranch Pipeline" job that takes some time such as: {code:java} pipeline { agent any stages { stage('only'){ steps { checkout scm sh 'sleep 300' } } } }{code} This gets automatically detected by the pipeline and executed. This can also be kicked off manually by going into the job and clicking the run button beside the branch (e.g. 'master'). If this button is pushed twice in succession (or two branches are committed at the same time) two instances of the job will run. If there is a single build executor that these jobs can run on (I have only seen this when the executor is separate from the "master" node), the first will start running. The second will do some pipeline action to identify what needs to be run, but then wait in the queue for the first job to complete. Once the first job has completed the second job will run to completion. For the above job, the first job will show as taking the appropriate five minutes. However, the second job will show as taking ten minutes....it is including the time it was waiting for an executor! If this is done with many jobs, the time will include all the time that each job had to wait in the queue. When looking at the historical builds, it will look like it took ten minutes to execute even though it was five of waiting and five of executing. This will also affect the projected build time for the next job run. This also shows "10 minutes building on an executor" if the metrics plugin is installed. The solution to this is to not include the time spent waiting for an executor in the build time recorded. I believe the matrix jobs had a similar issue that was fixed in Issue #8112. |
Roy
made changes -
| Priority | Original: Minor [ 4 ] | New: Blocker [ 1 ] |
| Comment | [ If possible, consider configuring more executors on your Jenkins instance so that parallel builds can run simultaneously. This will reduce the time spent waiting in the queue. If you have certain jobs that need to be prioritized over others, you can set job priorities or adjust the job scheduling strategy to ensure that the most important jobs are run first. Look into Jenkins plugins that may help with scheduling and queuing of jobs, such as the Throttle Concurrent Builds plugin. This plugin can help control the number of concurrent builds running and waiting. Consider separating the execution environment for jobs. For example, by using different nodes for different branches or projects, you can isolate jobs from each other and reduce queuing times. The metrics plugin and other build time tracking tools may need to be adjusted to account for the queue time separately from the execution time. Check the settings or configuration options for the plugins you are using to see if there are ways to customize the metrics. ] |
| Description |
Original:
The issue occours if you have a "Multibranch Pipeline" job that takes some time such as: {code:java} pipeline { agent any stages { stage('only'){ steps { checkout scm sh 'sleep 300' } } } }{code} This gets automatically detected by the pipeline and executed. This can also be kicked off manually by going into the job and clicking the run button beside the branch (e.g. 'master'). If this button is pushed twice in succession (or two branches are committed at the same time) two instances of the job will run. If there is a single build executor that these jobs can run on (I have only seen this when the executor is separate from the "master" node), the first will start running. The second will do some pipeline action to identify what needs to be run, but then wait in the queue for the first job to complete. Once the first job has completed the second job will run to completion. For the above job, the first job will show as taking the appropriate five minutes. However, the second job will show as taking ten minutes....it is including the time it was waiting for an executor! If this is done with many jobs, the time will include all the time that each job had to wait in the queue. When looking at the historical builds, it will look like it took ten minutes to execute even though it was five of waiting and five of executing. This will also affect the projected build time for the next job run. This also shows "10 minutes building on an executor" if the metrics plugin is installed. The solution to this is to not include the time spent waiting for an executor in the build time recorded. I believe the matrix jobs had a similar issue that was fixed in Issue #8112. |
New:
The issue occours if you have a "Multibranch Pipeline" job that takes some time such as:
{code:java} pipeline { agent any stages { stage('only'){ steps { checkout scm sh 'sleep 300' } } } }{code} This gets automatically detected by the pipeline and executed. This can also be kicked off manually by going into the job and clicking the run button beside the branch (e.g. 'master'). If this button is pushed twice in succession (or two branches are committed at the same time) two instances of the job will run. If there is a single build executor that these jobs can run on (I have only seen this when the executor is separate from the "master" node), the first will start running. The second will do some pipeline action to identify what needs to be run, but then wait in the queue for the first job to complete. Once the first job has completed the second job will run to completion. For the above job, the first job will show as taking the appropriate five minutes. However, the second job will show as taking ten minutes....it is including the time it was waiting for an executor! If this is done with many jobs, the time will include all the time that each job had to wait in the queue. When looking at the historical builds, it will look like it took ten minutes to execute even though it was five of waiting and five of executing. This will also affect the projected build time for the next job run. This also shows "10 minutes building on an executor" if the metrics plugin is installed. The solution to this is to not include the time spent waiting for an executor in the build time recorded. I believe the matrix jobs had a similar issue that was fixed in Issue #8112. |