Details
-
Type:
Bug
-
Status: Resolved (View Workflow)
-
Priority:
Major
-
Resolution: Incomplete
-
Component/s: pipeline
-
Labels:
-
Environment:Jenkins v2.69, running on Ubuntu 16.04LTS, with agents running Windows 10 and Ubuntu. Pipeline v2.5, Durable Task v1.14
-
Similar Issues:
Description
Jenkins intermittently enters a state where:
- Any started pipeline jobs hang indefinitely on sh, bat, and sleep steps regardless of if they are executed on master or an agent, Unix or Windows systems
- Pipelines without sh/bat/sleep steps complete, but do not release their executors when completed
- Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works
- Jenkins is otherwise responsive
- *Update* Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released.
The following Pipeline script with "sleep"
node('NodeName') { echo 'Hello World' sleep(1) }
results in hanging at the sleep step...
[Pipeline] node
Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang
[Pipeline] {
[Pipeline] echo
Hello World
[Pipeline] sleep
Sleeping for 1 sec
This pipeline script completes,
node('NodeName') { echo 'Hello World' }
but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted.
*Update* Multiple ThreadDumps attached from described Jenkins state.
Attachments
Activity
Anshu Arya
created issue -
Anshu Arya
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Description |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. ThreadDump attached from described Jenkins state. |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. ThreadDump attached from described Jenkins state. |
Anshu Arya
made changes -
| Attachment | ThreadDump2.txt [ 39019 ] |
Anshu Arya
made changes -
| Description |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. ThreadDump attached from described Jenkins state. |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. **Update** Multiple ThreadDumps attached from described Jenkins state. |
Anshu Arya
made changes -
| Attachment | ThreadDump-Agent.txt [ 39021 ] |
Anshu Arya
made changes -
| Description |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. **Update** Multiple ThreadDumps attached from described Jenkins state. |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. **Update** Multiple ThreadDumps attached from described Jenkins state. |
Anshu Arya
made changes -
| Description |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. **Update** Multiple ThreadDumps attached from described Jenkins state. |
Jenkins intermittently enters a state where: # Any started pipeline jobs *hang indefinitely on sh, bat, and sleep* steps regardless of if they are executed on master or an agent, Unix or Windows systems # Pipelines without sh/bat/sleep steps complete, but *do not release their executors* when completed # Using "/stop" on the pipeline job is completely unresponsive, but "/term" and "/kill" works # Jenkins is otherwise responsive # **Update** Executing shell/batch commands with groovy using String.execute() does NOT hang, but at the end of the pipeline, the executor is still not released. The following Pipeline script with "sleep" {code:java} node('NodeName') { echo 'Hello World' sleep(1) } {code} results in hanging at the sleep step... {code:java} [Pipeline] node Running on NodeName in C:\Jenkins_work\workspace\test_sleep_hang [Pipeline] { [Pipeline] echo Hello World [Pipeline] sleep Sleeping for 1 sec {code} This pipeline script completes, {code:java} node('NodeName') { echo 'Hello World' }{code} but does not release the executor on "NodeName" and then other jobs are blocked from running on that node as executors are exhausted. **Update** Multiple ThreadDumps attached from described Jenkins state. |
| Resolution | Incomplete [ 4 ] | |
| Status | Open [ 1 ] | Resolved [ 5 ] |