• Type: Bug
• Resolution: Unresolved
• Priority: Major
• Component/s: core
• Labels:

  None
• Environment:

  Hide

  Jenkins Server OS: Windows Server 2012 R2 (64-bit)
  Slaves' OS: Windows Server 2008 R2 (64-bit), Windows 10 Enterprise 2016 LTSB (64-bit)
  Jenkins Version: 2.164.1
  Java: jre1.8.0_161
  Slaves launched via Java Web Start
  Web Browser: Chrome Version 73.0.3683.86 (64-bit)
  
  Plugins
  ace-editor 1.1 true
  analysis-core 1.95 true
  analysis-model-api 2.1.2 true
  ant 1.9 true
  antisamy-markup-formatter 1.5 true
  apache-httpcomponents-client-4-api 4.5.5-3.0 true
  audit2db 0.5 true
  authentication-tokens 1.3 true
  blueocean 1.13.2 true
  blueocean-autofavorite 1.2.3 true
  blueocean-bitbucket-pipeline 1.13.2 true
  blueocean-commons 1.13.2 true
  blueocean-config 1.13.2 true
  blueocean-core-js 1.13.2 true
  blueocean-dashboard 1.13.2 true
  blueocean-display-url 2.2.0 true
  blueocean-events 1.13.2 true
  blueocean-git-pipeline 1.13.2 true
  blueocean-github-pipeline 1.13.2 true
  blueocean-i18n 1.13.2 true
  blueocean-jira 1.13.2 true
  blueocean-jwt 1.13.2 true
  blueocean-personalization 1.13.2 true
  blueocean-pipeline-api-impl 1.13.2 true
  blueocean-pipeline-editor 1.13.2 true
  blueocean-pipeline-scm-api 1.13.2 true
  blueocean-rest 1.13.2 true
  blueocean-rest-impl 1.13.2 true
  blueocean-web 1.13.2 true
  bouncycastle-api 2.17 true
  branch-api 2.1.2 true
  build-pipeline-plugin 1.5.8 true
  build-timeout 1.19 true
  build-timestamp 1.0.3 true
  build-token-root 1.4 true
  built-on-column 1.1 true
  cloudbees-bitbucket-branch-source 2.4.2 true
  cloudbees-folder 6.7 true
  command-launcher 1.3 true
  conditional-buildstep 1.3.6 true
  config-file-provider 3.5 true
  credentials 2.1.18 true
  credentials-binding 1.18 true
  dashboard-view 2.10 true
  display-url-api 2.3.0 true
  docker-commons 1.13 true
  docker-workflow 1.17 true
  durable-task 1.29 true
  email-ext 2.65 true
  envinject 2.1.6 true
  envinject-api 1.5 true
  external-monitor-job 1.7 true
  favorite 2.3.2 true
  git 3.9.3 true
  git-client 2.7.6 true
  git-server 1.7 true
  github 1.29.4 true
  github-api 1.95 true
  github-branch-source 2.4.2 true
  gradle 1.30 true
  greenballs 1.15 true
  groovy 2.2 true
  handlebars 1.1.1 true
  handy-uri-templates-2-api 2.1.7-1.0 true
  htmlpublisher 1.18 true
  http_request 1.8.22 true
  icon-shim 2.0.3 true
  jackson2-api 2.9.8 true
  javadoc 1.5 true
  jdk-tool 1.2 true
  jenkins-design-language 1.13.2 true
  jenkins-multijob-plugin 1.32 true
  jira 3.0.5 true
  jobConfigHistory 2.19 true
  jquery 1.12.4-0 true
  jquery-detached 1.2.1 true
  jsch 0.1.55 true
  junit 1.27 true
  ldap 1.20 true
  lockable-resources 2.4 true
  log-parser 2.0 true
  mailer 1.23 true
  managed-scripts 1.4 true
  mapdb-api 1.0.9.0 true
  matrix-auth 2.3 true
  matrix-project 1.14 true
  maven-plugin 3.2 true
  mercurial 2.5 true
  momentjs 1.1.1 true
  nodelabelparameter 1.7.2 true
  notification 1.13 true
  pam-auth 1.4 true
  Parameterized-Remote-Trigger 3.0.7 true
  parameterized-trigger 2.35.2 true
  pipeline-build-step 2.7 true
  pipeline-github-lib 1.0 true
  pipeline-graph-analysis 1.9 true
  pipeline-input-step 2.9 true
  pipeline-milestone-step 1.3.1 true
  pipeline-model-api 1.3.6 true
  pipeline-model-declarative-agent 1.1.1 true
  pipeline-model-definition 1.3.6 true
  pipeline-model-extensions 1.3.6 true
  pipeline-rest-api 2.10 true
  pipeline-stage-step 2.3 true
  pipeline-stage-tags-metadata 1.3.6 true
  pipeline-stage-view 2.10 true
  plain-credentials 1.5 true
  project-description-setter 1.2 true
  pubsub-light 1.12 true
  rebuild 1.29 true
  resource-disposer 0.12 true
  run-condition 1.2 true
  scm-api 2.3.0 true
  script-security 1.54 true
  sectioned-view 1.25 true
  sse-gateway 1.17 true
  ssh-credentials 1.15 true
  ssh-slaves 1.29.4 true
  structs 1.17 true
  subversion 2.12.1 true
  text-finder 1.10 true
  timestamper 1.9 true
  token-macro 2.7 true
  variant 1.2 true
  warnings 5.0.1 true
  warnings-ng 3.0.3 true
  windows-slaves 1.4 true
  workflow-aggregator 2.6 true
  workflow-api 2.33 true
  workflow-basic-steps 2.14 true
  workflow-cps 2.64 true
  workflow-cps-global-lib 2.13 true
  workflow-durable-task-step 2.29 true
  workflow-job 2.32 true
  workflow-multibranch 2.21 true
  workflow-scm-step 2.7 true
  workflow-step-api 2.19 true
  workflow-support 3.2 true
  ws-cleanup 0.37 true
  
  
  
  

  Show

  Jenkins Server OS: Windows Server 2012 R2 (64-bit) Slaves' OS: Windows Server 2008 R2 (64-bit), Windows 10 Enterprise 2016 LTSB (64-bit) Jenkins Version: 2.164.1 Java: jre1.8.0_161 Slaves launched via Java Web Start Web Browser: Chrome Version 73.0.3683.86 (64-bit) Plugins ace-editor 1.1 true analysis-core 1.95 true analysis-model-api 2.1.2 true ant 1.9 true antisamy-markup-formatter 1.5 true apache-httpcomponents-client-4-api 4.5.5-3.0 true audit2db 0.5 true authentication-tokens 1.3 true blueocean 1.13.2 true blueocean-autofavorite 1.2.3 true blueocean-bitbucket-pipeline 1.13.2 true blueocean-commons 1.13.2 true blueocean-config 1.13.2 true blueocean-core-js 1.13.2 true blueocean-dashboard 1.13.2 true blueocean-display-url 2.2.0 true blueocean-events 1.13.2 true blueocean-git-pipeline 1.13.2 true blueocean-github-pipeline 1.13.2 true blueocean-i18n 1.13.2 true blueocean-jira 1.13.2 true blueocean-jwt 1.13.2 true blueocean-personalization 1.13.2 true blueocean-pipeline-api-impl 1.13.2 true blueocean-pipeline-editor 1.13.2 true blueocean-pipeline-scm-api 1.13.2 true blueocean-rest 1.13.2 true blueocean-rest-impl 1.13.2 true blueocean-web 1.13.2 true bouncycastle-api 2.17 true branch-api 2.1.2 true build-pipeline-plugin 1.5.8 true build-timeout 1.19 true build-timestamp 1.0.3 true build-token-root 1.4 true built-on-column 1.1 true cloudbees-bitbucket-branch-source 2.4.2 true cloudbees-folder 6.7 true command-launcher 1.3 true conditional-buildstep 1.3.6 true config-file-provider 3.5 true credentials 2.1.18 true credentials-binding 1.18 true dashboard-view 2.10 true display-url-api 2.3.0 true docker-commons 1.13 true docker-workflow 1.17 true durable-task 1.29 true email-ext 2.65 true envinject 2.1.6 true envinject-api 1.5 true external-monitor-job 1.7 true favorite 2.3.2 true git 3.9.3 true git-client 2.7.6 true git-server 1.7 true github 1.29.4 true github-api 1.95 true github-branch-source 2.4.2 true gradle 1.30 true greenballs 1.15 true groovy 2.2 true handlebars 1.1.1 true handy-uri-templates-2-api 2.1.7-1.0 true htmlpublisher 1.18 true http_request 1.8.22 true icon-shim 2.0.3 true jackson2-api 2.9.8 true javadoc 1.5 true jdk-tool 1.2 true jenkins-design-language 1.13.2 true jenkins-multijob-plugin 1.32 true jira 3.0.5 true jobConfigHistory 2.19 true jquery 1.12.4-0 true jquery-detached 1.2.1 true jsch 0.1.55 true junit 1.27 true ldap 1.20 true lockable-resources 2.4 true log-parser 2.0 true mailer 1.23 true managed-scripts 1.4 true mapdb-api 1.0.9.0 true matrix-auth 2.3 true matrix-project 1.14 true maven-plugin 3.2 true mercurial 2.5 true momentjs 1.1.1 true nodelabelparameter 1.7.2 true notification 1.13 true pam-auth 1.4 true Parameterized-Remote-Trigger 3.0.7 true parameterized-trigger 2.35.2 true pipeline-build-step 2.7 true pipeline-github-lib 1.0 true pipeline-graph-analysis 1.9 true pipeline-input-step 2.9 true pipeline-milestone-step 1.3.1 true pipeline-model-api 1.3.6 true pipeline-model-declarative-agent 1.1.1 true pipeline-model-definition 1.3.6 true pipeline-model-extensions 1.3.6 true pipeline-rest-api 2.10 true pipeline-stage-step 2.3 true pipeline-stage-tags-metadata 1.3.6 true pipeline-stage-view 2.10 true plain-credentials 1.5 true project-description-setter 1.2 true pubsub-light 1.12 true rebuild 1.29 true resource-disposer 0.12 true run-condition 1.2 true scm-api 2.3.0 true script-security 1.54 true sectioned-view 1.25 true sse-gateway 1.17 true ssh-credentials 1.15 true ssh-slaves 1.29.4 true structs 1.17 true subversion 2.12.1 true text-finder 1.10 true timestamper 1.9 true token-macro 2.7 true variant 1.2 true warnings 5.0.1 true warnings-ng 3.0.3 true windows-slaves 1.4 true workflow-aggregator 2.6 true workflow-api 2.33 true workflow-basic-steps 2.14 true workflow-cps 2.64 true workflow-cps-global-lib 2.13 true workflow-durable-task-step 2.29 true workflow-job 2.32 true workflow-multibranch 2.21 true workflow-scm-step 2.7 true workflow-step-api 2.19 true workflow-support 3.2 true ws-cleanup 0.37 true

[JENKINS-56767] Unexpected exception occurred while performing online-node command

Chance Davies created issue - 2019-03-26 20:48

Chance Davies made changes - 2019-03-26 20:49

Description

Original: Multiple issues with launching well-known good jobs on slave nodes have been occurring after updating from Jenkins version 2.150.3 to 2.164.1. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days, and finally ceased only when the Jenkins version was downgraded to our previous version (2.150.3.) Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

New: Multiple issues with launching well-known good jobs on slave nodes have been occurring after updating from Jenkins version 2.150.3 to 2.164.1. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days, and finally ceased only when the Jenkins version was downgraded to our previous version (2.150.3.) No workaround was found for this issue, and only this downgrade was able to resolve our issues.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

Chance Davies made changes - 2019-03-26 21:15

Description

Original: Multiple issues with launching well-known good jobs on slave nodes have been occurring after updating from Jenkins version 2.150.3 to 2.164.1. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days, and finally ceased only when the Jenkins version was downgraded to our previous version (2.150.3.) No workaround was found for this issue, and only this downgrade was able to resolve our issues.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

New: Multiple issues with launching well-known good jobs on slave nodes have been occurring. Downgrading from Jenkins version 2.150.3 to 2.164.1. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days, and finally ceased only when the Jenkins version was downgraded to our previous version (2.150.3.) No workaround was found for this issue, and only this downgrade was able to resolve our issues.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

Chance Davies made changes - 2019-03-26 22:00

Description

Original: Multiple issues with launching well-known good jobs on slave nodes have been occurring. Downgrading from Jenkins version 2.150.3 to 2.164.1. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days, and finally ceased only when the Jenkins version was downgraded to our previous version (2.150.3.) No workaround was found for this issue, and only this downgrade was able to resolve our issues.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

New: Multiple issues with launching well-known good jobs on slave nodes have been occurring. Downgrading from Jenkins version 2.164.1 to 2.150.3 unfortunately does not appear to resolve the issue. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days and randomly started about a week ago. No workaround was found for this issue, and a downgrade was unable to resolve our issues.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

Chance Davies made changes - 2019-03-26 22:59

Description

Original: Multiple issues with launching well-known good jobs on slave nodes have been occurring. Downgrading from Jenkins version 2.164.1 to 2.150.3 unfortunately does not appear to resolve the issue. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days and randomly started about a week ago. No workaround was found for this issue, and a downgrade was unable to resolve our issues.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

New: Multiple issues with launching well-known good jobs on slave nodes have been occurring intermittently (<50%.) Downgrading from Jenkins version 2.164.1 to 2.150.3 unfortunately does not appear to resolve the issue. Specifically:  * Jobs failing due to "{color:#de350b}ERROR: Unexpected exception occurred while performing online-node command. java.net.SocketTimeoutException: Read timed out{color}"  * Jobs failing due to "{color:#de350b}ERROR: (job name) aborted.{color}" with no record of any user abort or reason for abort  * Jobs failing due to "{color:#de350b}FATAL: command execution failed java.nio.channels.ClosedChannelException{color}" This was reproduced over the course of multiple days and randomly started about a week ago. No workaround was found for this issue, and a downgrade was unable to resolve the issue.  Notepad documents containing the entirety of each of these errors from our job output have been attached, but if any other logs are needed, please let me know. These jobs provide a variety of functions, mostly in the way of running simple batch commands, disconnecting the node, then reconnecting only after a ping response fails for a time, then succeeds. The output provided should provide more information.

Chance Davies made changes - 2019-03-27 16:56

Attachment

Original: Unexpected exception occurred while performing online-node command.txt [ 46564 ]

Chance Davies made changes - 2019-03-27 16:56

Attachment

Original: Job Randomly Aborted.txt [ 46566 ]

Chance Davies made changes - 2019-03-27 16:56

Attachment

Original: java.nio.channels.ClosedChannelException.txt [ 46565 ]

Collapse comment: Chance Davies added a comment - 2019-03-27 21:00

Chance Davies added a comment - 2019-03-27 21:00

Please note: My company has recommended I remove the logs which were originally attached to this bug. I can still provide specifics and logs which do not contain system names or IPs upon request. 

Expand comment: Chance Davies added a comment - 2019-03-27 21:00

Chance Davies added a comment - 2019-03-27 21:00 Please note: My company has recommended I remove the logs which were originally attached to this bug. I can still provide specifics and logs which do not contain system names or IPs upon request. 

Thomas de Grenier de Latour made changes - 2020-02-15 22:45

Link

New: This issue relates to JENKINS-61103 [ JENKINS-61103 ]

Load more newer events