Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42940

Timeout step hangs after restart if timeout occurred, but enclosed block did not exit yet


    • workflow-basic-steps 2.20

      In case the timeout occurs, and Jenkins is restarted during the grace period if waits for the inner block to terminate, then the build hangs forever with this exception in the Jenkins log:

      2020-03-13 02:09:40.575+0000 [id=1502]  WARNING o.j.p.w.f.FlowExecutionList$ItemListenerImpl$1#onFailure: Failed to load CpsFlowExecution[Owner[devops-gate/master/blackbox-self-service/25907:devops-gate/master/blackbox-self-service #25907]]
              at org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution.cancel(TimeoutStepExecution.java:151)
              at org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution.setupTimer(TimeoutStepExecution.java:139)
              at org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution.onResume(TimeoutStepExecution.java:90)
              at org.jenkinsci.plugins.workflow.flow.FlowExecutionList$ItemListenerImpl$1.onSuccess(FlowExecutionList.java:185)
              at org.jenkinsci.plugins.workflow.flow.FlowExecutionList$ItemListenerImpl$1.onSuccess(FlowExecutionList.java:180)

      Reproducability of this issue relies on a block that does not immediately Exit. For example:

      node {
       timeout (time: 10, unit: 'SECONDS') {
         build job: 'hang2', parameters: [ new StringParameterValue('A','B') ], quietPeriod: 0

      with a second Pipeline Job hang2:

      retry(3) {
           sleep 300

      Creates this console log:

      Gestartet durch Benutzer RK
       [Pipeline] node
       Running on host in /$JENKINS_HOME/workspace/hang
       [Pipeline] {
       [Pipeline] timeout
       Timeout set to expire in 10 Sekunden
       [Pipeline] {
       [Pipeline] build (Building hang2)
       Scheduling project: hang2
       Starting building: hang2 #1
       Cancelling nested steps due to timeout
       Resuming build at Fri Mar 10 15:49:00 CET 2017 after Jenkins restart
       Waiting to resume hang #1|: ???
       Waiting to resume hang #1|: host ist offline
       Waiting to resume hang #1|: host ist offline
      Ready to run at Fri Mar 10 15:49:10 CET 2017
      Timeout expired 3,7 Sekunden ago

      ... and then it hangs forever.

      Reason: when onResume() is called, the timer is expired, so cancel() is called, and since it already tried to cancel, forcible is true, and then killer is null, causing an NPE.

      Fix: Check killer for null on line 94 in cancel() in TimeoutStepExecution().

      Rationale for Major, not minor bug: breaks restart resiliense.

            dnusbaum Devin Nusbaum
            rk R K
            1 Vote for this issue
            4 Start watching this issue