I've noticed this problem as well, it really makes "Prepare for shutdown" quite useless for us because it's tedious to have to hard kill our currently running jobs. The whole idea of "Prepare for shutdown" is to allow us to finish the currently running jobs, not freeze them where they are.

It's worth calling out that this has only happened in a recent upgrade of either Jenkins or one of the Pipeline plugins (Sorry it's vague but I'd estimate a change in the last 1-2 months)

Here is a workflow script to repro:

1. Create and start a job that uses the following code:

node( "testnode" )
{
    for( int i = 0; i < 10; i++ )
    {
        println "Sleep: ${i+1}"
        sleep(5)
    }
}

2. Activate "Prepare for Shutdown" mode while the job is running.

Expected Result: Job continues to run as it's already midway through execution.
Actual Result: Job freezes

srwalter this might not help you, but if you cancel "Prepare for Shutdown" and then on your job click "Pause/resume" to toggle the job into Paused, then Resumed states, the job will continue. Eg.

Sleep: 3
[Pipeline] sleep
Sleeping for 5 sec
Pausing
Resuming
[Pipeline] echo
Sleep: 4
[Pipeline] sleep
Sleeping for 5 sec
[Pipeline] echo
Sleep: 5

To me, #3 is really the issue here. Due to the nature of pipeline jobs (serialized, stoppable at just about any point), it doesn't bother me that #2 happens. But cancelling the shutdown mode (without a restart) should cause the jobs to start back up again, and it currently doesn't.

And it may go without saying, but we're seeing this issue too.

An effect of the fix of JENKINS-32015.

Freezing the build—that is, the Groovy part—is deliberate, to prevent far worse problems, since we do not know when the actual JVM exit is going to come (TBD if we can safely block TermMilestone.COMPLETED). But the possibility that this mode would subsequently be cancelled was never considered. Pause + Resume is the appropriate workaround.

The whole idea of "Prepare for shutdown" is to allow us to finish the currently running jobs

It is to allow you to finish currently running freestyle (Maven, matrix, …) builds. So if you /safeRestart Jenkins will restart as soon as any of those are completed, and running Pipeline builds will be left alone.

Any update on this issue? I have been seeing the same issue where pipeline jobs don't complete when in shutdown mode and the only way to stop the job is to do a hard kill.

I'm using Jenkins 2.66 and the pipelines plugin is version 2.5

Manual Pause & Resume might work around it; have not tried.

Not sure what the fix would look like. There is no event fired when quiet mode is cancelled AFAIK, so you would need to set up some kind of polling using Timer. At any rate, sounds like it should be easy enough to write a functional test for.

Given that pipelines are now a central feature of Jenkins would we be able to add an event for cancelling quiet mode (if one doesn't exist)? I feel like there might be other processes that could be interested in this and it would make for a much cleaner solution.

Seeing this on Jenkins 2.73.2 as well. Very odd that simple pipeline jobs totally hang just by entering quiet mode. It seems to cause them to hang in an unrecoverable way. We use "Quiet Down" to restart for upgrades to Jenkins and plugins in a graceful way, but very ironic and problematic that it is behaving very ungracefully by causing any in progress pipelines to get stuck.

Also a problem in Jenkins: 2.89.4, Pipeline: 2.5 

ad #2 pipeline stops/freezes instead of expected and documented (e.g. also in UI "Manage Jenkins > Prepare for Shutdown") finishing of job:

• this is surprising and unexpected considering also the documentation, but of course makes sense as described by Jesse.
• however, would it make sense to change that to e.g. finish execution of currently active stage like commented in JENKINS-49365 – and somehow signal that in the UI similar to current "(part)" suffix in "build executor status" to allow for gentle shutdown at well-defined points? And therefore, as mentioned in JENKINS-49365, also when docker.image(...).inside(...) is used...

ad #3 pipeline does not wake up after canceling the shutdown mode:

• yes, I can confirm: "pause" (also) after cancelling followed by "resume" works and would be an IMHO acceptable workaround – although I really would not mind if that would be "just" fixed, naively assuming that it is not that difficult to do?

While (resumable, i.e. non-docker.image(...).inside(...)-dependent) pipelines resume fine after a restart, there is a remainder in "build executor status" that (after the pipeline has finished) requires the workaround to abort it (the finished pipeline) by pressing the 'x' that in turn then asks "Are you sure you want to abort null?":

Closing in preference to the pre-existing issue that has the same result (sigh, still not resolved yet).

Any updates would be very appreciated. This bug is causing a lot of problems for our build system

This still happened to Jenkins v*2.138.4* LTS

(incorrect resolution)