-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
Operating System: linux containers -- jenkins/jenkins:2.73.2-alpine & jenkins/jnlp-slave
Jenkins Master's runtime parameters:
-Djenkins.install.runSetupWizard=false
-Dpermissive-script-security.enabled=no_security
-Dhudson.model.DownloadService.noSignatureCheck=true
-Dorg.apache.commons.jelly.tags.fmt.timeZone=America/Chicago
Slaves' runtime parameters:
(none)
Plugin versions:
ace-editor:1.1
ansicolor:0.5.2
ant:1.7
antisamy-markup-formatter:1.5
apache-httpcomponents-client-4-api:4.5.3-2.0
authentication-tokens:1.3
blueocean-autofavorite:1.0.0
blueocean-bitbucket-pipeline:1.3.0
blueocean-commons:1.3.0
blueocean-config:1.3.0
blueocean-dashboard:1.3.0
blueocean-display-url:2.1.0
blueocean-events:1.3.0
blueocean-git-pipeline:1.3.0
blueocean-github-pipeline:1.3.0
blueocean-i18n:1.3.0
blueocean-jira:1.3.0
blueocean-jwt:1.3.0
blueocean-personalization:1.3.0
blueocean-pipeline-api-impl:1.3.0
blueocean-pipeline-editor:1.3.0
blueocean-pipeline-scm-api:1.3.0
blueocean-rest-impl:1.3.0
blueocean-rest:1.3.0
blueocean-web:1.3.0
blueocean:1.3.0
bouncycastle-api:2.16.2
branch-api:2.0.14
build-timeout:1.19
cloudbees-bitbucket-branch-source:2.2.3
cloudbees-disk-usage-simple:0.9
cloudbees-folder:6.2.1
compress-artifacts:1.10
conditional-buildstep:1.3.6
copyartifact:1.38.1
credentials-binding:1.13
credentials:2.1.16
display-url-api:2.1.0
docker-commons:1.9
docker-workflow:1.13
durable-task:1.15
email-ext:2.60
external-monitor-job:1.7
favorite:2.3.0
git-client:2.5.0
git-server:1.7
git:3.6.0
github-api:1.89
github-branch-source:2.2.3
github-organization-folder:1.6
github:1.28.0
google-login:1.3
gradle:1.28
greenballs:1.15
groovy:2.0
handlebars:1.1.1
htmlpublisher:1.14
icon-shim:2.0.3
jackson2-api:2.8.7.0
javadoc:1.4
jira-steps:1.2.3
jira:2.4.2
job-dsl:1.66
jquery-detached:1.2.1
jsch:0.1.54.1
junit:1.21
kubernetes-pipeline-aggregator:1.5
kubernetes-pipeline-arquillian-steps:1.5
kubernetes-pipeline-devops-steps:1.5
kubernetes-pipeline-steps:1.5
kubernetes:1.1
ldap:1.17
logfilesizechecker:1.2
mailer:1.20
mapdb-api:1.0.9.0
matrix-auth:2.1
matrix-project:1.12
maven-plugin:3.0
mercurial:2.2
metrics:3.1.2.10
momentjs:1.1.1
pam-auth:1.3
parameterized-trigger:2.35.2
permissive-script-security:0.3
pipeline-aggregator-view:1.8
pipeline-build-step:2.5.1
pipeline-github-lib:1.0
pipeline-graph-analysis:1.5
pipeline-input-step:2.8
pipeline-milestone-step:1.3.1
pipeline-model-api:1.2.2
pipeline-model-declarative-agent:1.1.1
pipeline-model-definition:1.2.2
pipeline-model-extensions:1.2.2
pipeline-rest-api:2.9
pipeline-stage-step:2.2
pipeline-stage-tags-metadata:1.2.2
pipeline-stage-view:2.9
plain-credentials:1.4
powershell:1.3
pubsub-light:1.12
resource-disposer:0.8
run-condition:1.0
scm-api:2.2.3
script-security:1.34
sse-gateway:1.15
ssh-agent:1.15
ssh-credentials:1.13
ssh-slaves:1.21
structs:1.10
subversion:2.9
timestamper:1.8.8
token-macro:2.3
variant:1.1
windows-slaves:1.3.1
workflow-aggregator:2.5
workflow-api:2.22
workflow-basic-steps:2.6
workflow-cps-global-lib:2.9
workflow-cps:2.41
workflow-durable-task-step:2.17
workflow-job:2.15
workflow-multibranch:2.16
workflow-scm-step:2.6
workflow-step-api:2.13
workflow-support:2.16
ws-cleanup:0.34
Reverse proxy? Yes. If configuration information for this is necessary, please let me know.
Jenkins Master runs in Kubernetes, so do the ephemeral slaves.
Web browser? Chrome 61.0.3163.100 (Official Build) (64-bit)Operating System: linux containers -- jenkins/jenkins:2.73.2-alpine & jenkins/jnlp-slave Jenkins Master's runtime parameters: -Djenkins.install.runSetupWizard=false -Dpermissive-script-security.enabled=no_security -Dhudson.model.DownloadService.noSignatureCheck=true -Dorg.apache.commons.jelly.tags.fmt.timeZone=America/Chicago Slaves' runtime parameters: (none) Plugin versions: ace-editor:1.1 ansicolor:0.5.2 ant:1.7 antisamy-markup-formatter:1.5 apache-httpcomponents-client-4-api:4.5.3-2.0 authentication-tokens:1.3 blueocean-autofavorite:1.0.0 blueocean-bitbucket-pipeline:1.3.0 blueocean-commons:1.3.0 blueocean-config:1.3.0 blueocean-dashboard:1.3.0 blueocean-display-url:2.1.0 blueocean-events:1.3.0 blueocean-git-pipeline:1.3.0 blueocean-github-pipeline:1.3.0 blueocean-i18n:1.3.0 blueocean-jira:1.3.0 blueocean-jwt:1.3.0 blueocean-personalization:1.3.0 blueocean-pipeline-api-impl:1.3.0 blueocean-pipeline-editor:1.3.0 blueocean-pipeline-scm-api:1.3.0 blueocean-rest-impl:1.3.0 blueocean-rest:1.3.0 blueocean-web:1.3.0 blueocean:1.3.0 bouncycastle-api:2.16.2 branch-api:2.0.14 build-timeout:1.19 cloudbees-bitbucket-branch-source:2.2.3 cloudbees-disk-usage-simple:0.9 cloudbees-folder:6.2.1 compress-artifacts:1.10 conditional-buildstep:1.3.6 copyartifact:1.38.1 credentials-binding:1.13 credentials:2.1.16 display-url-api:2.1.0 docker-commons:1.9 docker-workflow:1.13 durable-task:1.15 email-ext:2.60 external-monitor-job:1.7 favorite:2.3.0 git-client:2.5.0 git-server:1.7 git:3.6.0 github-api:1.89 github-branch-source:2.2.3 github-organization-folder:1.6 github:1.28.0 google-login:1.3 gradle:1.28 greenballs:1.15 groovy:2.0 handlebars:1.1.1 htmlpublisher:1.14 icon-shim:2.0.3 jackson2-api:2.8.7.0 javadoc:1.4 jira-steps:1.2.3 jira:2.4.2 job-dsl:1.66 jquery-detached:1.2.1 jsch:0.1.54.1 junit:1.21 kubernetes-pipeline-aggregator:1.5 kubernetes-pipeline-arquillian-steps:1.5 kubernetes-pipeline-devops-steps:1.5 kubernetes-pipeline-steps:1.5 kubernetes:1.1 ldap:1.17 logfilesizechecker:1.2 mailer:1.20 mapdb-api:1.0.9.0 matrix-auth:2.1 matrix-project:1.12 maven-plugin:3.0 mercurial:2.2 metrics:3.1.2.10 momentjs:1.1.1 pam-auth:1.3 parameterized-trigger:2.35.2 permissive-script-security:0.3 pipeline-aggregator-view:1.8 pipeline-build-step:2.5.1 pipeline-github-lib:1.0 pipeline-graph-analysis:1.5 pipeline-input-step:2.8 pipeline-milestone-step:1.3.1 pipeline-model-api:1.2.2 pipeline-model-declarative-agent:1.1.1 pipeline-model-definition:1.2.2 pipeline-model-extensions:1.2.2 pipeline-rest-api:2.9 pipeline-stage-step:2.2 pipeline-stage-tags-metadata:1.2.2 pipeline-stage-view:2.9 plain-credentials:1.4 powershell:1.3 pubsub-light:1.12 resource-disposer:0.8 run-condition:1.0 scm-api:2.2.3 script-security:1.34 sse-gateway:1.15 ssh-agent:1.15 ssh-credentials:1.13 ssh-slaves:1.21 structs:1.10 subversion:2.9 timestamper:1.8.8 token-macro:2.3 variant:1.1 windows-slaves:1.3.1 workflow-aggregator:2.5 workflow-api:2.22 workflow-basic-steps:2.6 workflow-cps-global-lib:2.9 workflow-cps:2.41 workflow-durable-task-step:2.17 workflow-job:2.15 workflow-multibranch:2.16 workflow-scm-step:2.6 workflow-step-api:2.13 workflow-support:2.16 ws-cleanup:0.34 Reverse proxy? Yes. If configuration information for this is necessary, please let me know. Jenkins Master runs in Kubernetes, so do the ephemeral slaves. Web browser? Chrome 61.0.3163.100 (Official Build) (64-bit)
Our situation
We run Jenkins Master and ephemeral slaves in kubernetes clusters and love it. However we quite regularly run into stalled pipelines that wait indefinitely for a kubernetes ephemeral slave to come back online, which will never happen. I have no 100% reproducible steps because this does not happen every time. Instead it happens on average once out of 15 slaves. A slave will inexplicably disconnect/crash/be-decommissioned and the Jenkins pipeline using it thinks, "Ok, durable task means I wait for the slave that just went offline to come back online... forever." And so it does. :/
Someone else has opened a ticket with Jenkins Master team requesting an option to flag a job/task as not durable, however they do make clear that this behavior is something that needs to be addressed by the plugin as well.
The Story "Requirement"
This may be a bug or a feature/improvement request, depending on how your team has intended the plugin to function. I have flagged this as a bug because I do not believe this behavior is intended.
Kubernetes slaves should fail a pipeline step/task that is running on it whenever the slave goes offline.
- duplicates
-
-
- Resolved
-
-
JENKINS-49707 Auto retry for elastic agents after channel closure
-
- Resolved
-
