At any time while a job is running and the master pod is killed the new pod will come up and the slaves will not be able to reconnect to it and finish the job. Instead the job times out after 300s and says the executor is assumed to be never coming back. Looking at the Jenkins master logs it appears that during the initialization of the master it tries to provision the slave but fails because it already exists (looking at the source code for provisioning at this link (line 221) it looks like it deletes the slave from the master's list of executor nodes). Then when the slave tries to reconnect once the master is back up the master rejects the slave's connection and the slave pod goes into an error state.

To reproduce: start a Jenkins job that sleeps 300, then sleeps another 300. During the first sleep, delete the master pod. The job will never get to the second sleep, and will time out and end in FAILURE.

Interesting note, if Jenkins master is restarted as part of a rolling upgrade of the master deployment, any slaves created from kubernetes plugin 0.11 that hang around while the master updates to the version with 1.1 will reconnect successfully after the restart.

This issue does not occur when using kubernetes plugin 1.0.