Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49365

Resuming of pipelines vs. docker.image(...).inside(...)?

      (Alternative way of asking for help; cf. https://groups.google.com/forum/#!msg/jenkinsci-users/xMxrpRJHIoA/KQoWVbnAAwAJ for my initial attempt)

      When playing around with resuming of pipelines after restarting the Jenkins master (note - not sure if of relevance though: it is a single jenkins master without any slaves setup) during pipeline execution, I stumbled over the the following problem when the restart happens in the course of executing a docker.image(...).inside(...) closure:

      java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image
          at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
          at java.lang.Class.forName0(Native Method)
          at java.lang.Class.forName(Class.java:348)
          at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
          at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:948)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1255)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
          at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
          at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149)
          at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135)
          at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53)
          at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
          at java.util.HashMap.readObject(HashMap.java:1409)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:309)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1644)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
          at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
          at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
          at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:742)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:735)
          at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158)
          at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40)
          at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48)
          at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112)
          at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      Caused: java.io.IOException: Failed to load build state
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:814)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:812)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:861)
          at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      

      Here is the build log right before the problem and including the resume logs:

       [Pipeline] sh
       [ACME-Pipeline (Branch)-WS] Running shell script
       + docker inspect -f . ACME/phpunit:5
       .
       [Pipeline] withDockerContainer
       Jenkins does not seem to be running inside a container
       $ docker run -t -d -u 10112:10005 --entrypoint -w "/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS" -v "/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS:/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS:rw,z" -v "/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS@tmp:/var/lib/jenkins/workspace/ACME-Pipeline (Branch)-WS@tmp:rw,z" -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** --entrypoint cat ACME/phpunit:5
       [Pipeline] \{
       [Pipeline] sh
       [ACME-Pipeline (Branch)-WS] Running shell script
       ...
       [Pipeline] sh
       [ACME-Pipeline (Branch)-WS] Running shell script
       + ./gradlew --no-daemon -PphpUnitXml=phpunit-result.xml executePlainPHPUnitTests
       :deleteDependencyCache UP-TO-DATE
       :executePlainPHPUnitTests
       PHPUnit 6.5.5 by Sebastian Bergmann and contributors.
      
      ............................................................... 63 / 484 ( 13%)
       ............................................................... 126 / 484 ( 26%)
       ............................................................... 189 / 484 ( 39%)
       ............................................................... 252 / 484 ( 52%)
       ............................................................... 315 / 484 ( 65%)
       ............................................................... 378 / 484 ( 78%)
       ............................................................... 441 / 484 ( 91%)
       ........................................... 484 / 484 (100%)
      
      Time: 10.73 seconds, Memory: 78.00MB
      
      OK (484 tests, 927 assertions)
      
      BUILD SUCCESSFUL in 15s
       2 actionable tasks: 1 executed, 1 up-to-date
       Resuming build at Wed Jan 24 15:57:45 CET 2018 after Jenkins restart
       Waiting to resume part of ACME-Pipeline (Branch) 20180124-155522-revUNKNOWN: Jenkins is about to shut down
       Waiting to resume part of ACME-Pipeline (Branch) 20180124-155522-revUNKNOWN: Jenkins is about to shut down
       Waiting to resume part of ACME-Pipeline (Branch) 20180124-155522-revUNKNOWN: Jenkins is about to shut down
       [Pipeline] End of Pipeline
       java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image
       at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      

      Here is a thread dump after "Manage Jenkins > Prepare for Shutdown" + waiting for the finishing of the "sh" step using gradle to execute PHPUnit and right before restarting the Jenkins master.

      Thread #18
          at com.ACME.php.PHPUnitViaGradle.execute([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:30|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:30])
          at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:129|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:129])
          at DSL.withDockerContainer(Native Method)
          at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:128|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:128])
          at org.jenkinsci.plugins.docker.workflow.Docker.node(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:63|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:63])
          at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:[file:/var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:116|file:///var/lib/jenkins/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:116])
          at com.ACME.php.PHPUnitViaGradle.execute([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:26|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/php/PHPUnitViaGradle.groovy:26])
          at ACME.executePHPUnitTests(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:191)
          at WorkflowScript.run(WorkflowScript:65)
          at com.ACME.Workspace.prepare([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:53|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:53])
          at DSL.ws(Native Method)
          at com.ACME.Workspace.prepare([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:30|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Workspace.groovy:30])
          at ACME.wsWithArtifacts(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:164)
          at WorkflowScript.run(WorkflowScript:64)
          at DSL.node(running on )
          at WorkflowScript.run(WorkflowScript:55)
          at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:76)
          at DSL.stage(Native Method)
          at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:67)
          at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:57)
          at ACME.skipableStage(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:52)
          at WorkflowScript.run(WorkflowScript:53)
          at com.ACME.Build.build([file:/var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Build.groovy:42|file:///var/lib/jenkins/jobs/ACME-Pipeline%20(Branch)/builds/660/libs/ACME-shared-library/src/com/ACME/Build.groovy:42])
          at ACME.build(/var/lib/jenkins/jobs/ACME-Pipeline (Branch)/builds/660/libs/ACME-shared-library/vars/ACME.groovy:48)
          at WorkflowScript.run(WorkflowScript:19)
      

      Not sure if it is relevant, but the Jenkins setup is very up to date: Jenins v2.103 with very recent update of these plugins:

          ---> Plugins installed:
          ...
          workflow-api:2.25
          workflow-cps:2.43
          workflow-durable-task-step:2.18
          workflow-job:2.17
          workflow-multibranch:2.17
          workflow-support:2.17
      

      ..., and thus this output at the top of the build log:

          Running in Durability level: MAX_SURVIVABILITY
      

      Is this supposed to work? If so, what am I missing?
      Or can it work at all?

          [JENKINS-49365] Resuming of pipelines vs. docker.image(...).inside(...)?

          Reinhold Füreder added a comment - - edited

          JENKINS-38316 covers both "actually items", that is the (surprising) pipeline pausing/freezing instead of the expected finish the job due to "Manage Jenkins > Prepare for Shutdown", as well as the waking up...

          Reinhold Füreder added a comment - - edited JENKINS-38316 covers both "actually items", that is the (surprising) pipeline pausing/freezing instead of the expected finish the job due to "Manage Jenkins > Prepare for Shutdown", as well as the waking up...

          Jesse Glick added a comment -

          Yes it is supposed to work. I have not seen the error you mention before; offhand I would guess it has something to do with the use of a library. Best to avoid use of the DSL and call the withDockerContainer step directly. (Better still to not use this step at all and call sh steps directly, running docker build or docker-compose or whatever would make sense for your task if you had never heard of Jenkins.)

          Jesse Glick added a comment - Yes it is supposed to work . I have not seen the error you mention before; offhand I would guess it has something to do with the use of a library. Best to avoid use of the DSL and call the withDockerContainer step directly. (Better still to not use this step at all and call sh steps directly, running docker build or docker-compose or whatever would make sense for your task if you had never heard of Jenkins.)

          jglick Thanks to parts of your comment ("Yes it is supposed to work. I have not seen the error you mention before") I started to re-produce the problem in a small context in order to later on try your suggestions (e.g. "avoid use of DSL" or even "calling sh steps directly") more or less full of hope... BUT ... and this is indeed very good news: I not only failed to re-produce the problem in the small context (even without "use of a library"), but also in my original real-world scenario!!!

          And in the latter I actually remembered having no problems to re-produce it back then...

          (svanoort And even better, during my reproduction efforts I also never had to use the pause + resume workaround after cancelling the shutdown...)

          Not sure if it is relevant or could explain the problem, but of course there were a range of Jenkins (core) and especially (workflow/cps related) Jenkins plugins updates since then:

          • Jenkins core 2.124 (was: 2.103)
          • Plugins installed:
                workflow-api:2.27 (was: 2.25)
                workflow-cps:2.53 (was: 2.43)
                workflow-durable-task-step:2.19 (was: 2.18)
                workflow-job:2.21 (was: 2.17)
                workflow-support:2.18 (was: 2.17)
            

          => I'll resolve this issue with "cannot reproduce" (or so)...

          Reinhold Füreder added a comment - jglick Thanks to parts of your comment ("Yes it is supposed to work. I have not seen the error you mention before") I started to re-produce the problem in a small context in order to later on try your suggestions (e.g. "avoid use of DSL" or even "calling sh steps directly") more or less full of hope... BUT ... and this is indeed very good news: I not only failed to re-produce the problem in the small context (even without "use of a library"), but also in my original real-world scenario!!! And in the latter I actually remembered having no problems to re-produce it back then... ( svanoort And even better, during my reproduction efforts I also never had to use the pause + resume workaround after cancelling the shutdown...) Not sure if it is relevant or could explain the problem, but of course there were a range of Jenkins (core) and especially (workflow/cps related) Jenkins plugins updates since then: Jenkins core 2.124 (was: 2.103) Plugins installed: workflow-api:2.27 (was: 2.25) workflow-cps:2.53 (was: 2.43) workflow-durable-task-step:2.19 (was: 2.18) workflow-job:2.21 (was: 2.17) workflow-support:2.18 (was: 2.17) => I'll resolve this issue with "cannot reproduce" (or so)...

          Sam Van Oort added a comment -

          reinholdfuereder Any issue that can no longer be reproduced after apply updates is one I'll count as a win!

          Sam Van Oort added a comment - reinholdfuereder Any issue that can no longer be reproduced after apply updates is one I'll count as a win!

          Absolutely!

          Reinhold Füreder added a comment - Absolutely!

          Unfortunately it ("java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image") happend now (once) again:

          ...
          [Pipeline] stage
          [Pipeline] { (docker-other-tests)
          [Pipeline] node
          Running on Jenkins in /var/lib/jenkins/workspace/ACME-Pipeline
          [Pipeline] {
          [Pipeline] ws
          Running in /var/lib/jenkins/workspace/ACME-Pipeline-WS
          [Pipeline] {
          [Pipeline] sh
          [ACME-Pipeline-WS] Running shell script
          + find . -mindepth 1 -delete
          [Pipeline] unstash
          [Pipeline] sh
          [ACME-Pipeline-WS] Running shell script
          + unzip -q acme.zip
          [Pipeline] pwd
          [Pipeline] unstash
          [Pipeline] sh
          [ACME-Pipeline-WS] Running shell script
          + unzip -q acme-unittests.zip
          [Pipeline] pwd
          [Pipeline] sh
          [ACME-Pipeline-WS] Running shell script
          + docker inspect -f . acme/phpunit:5
          .
          [Pipeline] withDockerContainer
          Jenkins does not seem to be running inside a container
          $ docker run -t -d -u 10112:10005 --dns=10.1.114.5 -w /var/lib/jenkins/workspace/ACME-Pipeline-WS -v /var/lib/jenkins/workspace/ACME-Pipeline-WS:/var/lib/jenkins/workspace/ACME-Pipeline-WS:rw,z -v /var/lib/jenkins/workspace/ACME-Pipeline-WS@tmp:/var/lib/jenkins/workspace/ACME-Pipeline-WS@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** acme/phpunit:5 cat
          $ docker top 5e3d99376e810429e9d57ce61201af771f2dc3d2feb3cd21335398eadbfd3370 -eo pid,comm
          [Pipeline] {
          ...
          [Pipeline] sh
          [ACME-Pipeline-WS] Running shell script
          + ./gradlew --no-daemon -PphpUnitXml=phpunit-result.xml executeOtherPHPUnitTests
          > Task :deleteDependencyCache UP-TO-DATE
          
          > Task :executeOtherPHPUnitTests
          Execution according to PHPUnit XML configuration file './unittests/AllOtherPHPUnitTests.xml'...
          PHPUnit 6.5.8 by Sebastian Bergmann and contributors.
          
          .............................................                     45 / 45 (100%)
          
          Time: 1.62 minutes, Memory: 82.00MB
          
          OK (45 tests, 89 assertions)
          
          BUILD SUCCESSFUL in 1m 43s
          2 actionable tasks: 1 executed, 1 up-to-date
          Resuming build at Thu Jun 28 08:29:44 CEST 2018 after Jenkins restart
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: ???
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Resuming build at Thu Jun 28 08:32:27 CEST 2018 after Jenkins restart
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: ???
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Resuming build at Thu Jun 28 08:34:59 CEST 2018 after Jenkins restart
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: ???
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down
          [Pipeline] End of Pipeline
          java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image
          	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
          	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
          	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
          	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
          	at java.lang.Class.forName0(Native Method)
          	at java.lang.Class.forName(Class.java:348)
          	at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131)
          	at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:948)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1255)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
          	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
          	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          	at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149)
          	at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135)
          	at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53)
          	at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
          	at java.util.HashMap.readObject(HashMap.java:1409)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:309)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1644)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)
          	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
          	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
          	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.lambda$readObject$0(RiverReader.java:250)
          	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
          	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:237)
          	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:250)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:782)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:775)
          	at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48)
          	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          Caused: java.io.IOException: Failed to load build state
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:854)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:852)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:906)
          	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE
          

          Is that a kind of timing issue: depending on which point during the sh step (that is running inside the Docker container) the Jenkins master restart is happening (of course I had used the "Manage Jenkins > Prepare for Shutdown" before)!?

          Reinhold Füreder added a comment - Unfortunately it (" java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image ") happend now (once) again: ... [Pipeline] stage [Pipeline] { (docker-other-tests) [Pipeline] node Running on Jenkins in /var/lib/jenkins/workspace/ACME-Pipeline [Pipeline] { [Pipeline] ws Running in /var/lib/jenkins/workspace/ACME-Pipeline-WS [Pipeline] { [Pipeline] sh [ACME-Pipeline-WS] Running shell script + find . -mindepth 1 -delete [Pipeline] unstash [Pipeline] sh [ACME-Pipeline-WS] Running shell script + unzip -q acme.zip [Pipeline] pwd [Pipeline] unstash [Pipeline] sh [ACME-Pipeline-WS] Running shell script + unzip -q acme-unittests.zip [Pipeline] pwd [Pipeline] sh [ACME-Pipeline-WS] Running shell script + docker inspect -f . acme/phpunit:5 . [Pipeline] withDockerContainer Jenkins does not seem to be running inside a container $ docker run -t -d -u 10112:10005 --dns=10.1.114.5 -w /var/lib/jenkins/workspace/ACME-Pipeline-WS -v /var/lib/jenkins/workspace/ACME-Pipeline-WS:/var/lib/jenkins/workspace/ACME-Pipeline-WS:rw,z -v /var/lib/jenkins/workspace/ACME-Pipeline-WS@tmp:/var/lib/jenkins/workspace/ACME-Pipeline-WS@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** acme/phpunit:5 cat $ docker top 5e3d99376e810429e9d57ce61201af771f2dc3d2feb3cd21335398eadbfd3370 -eo pid,comm [Pipeline] { ... [Pipeline] sh [ACME-Pipeline-WS] Running shell script + ./gradlew --no-daemon -PphpUnitXml=phpunit-result.xml executeOtherPHPUnitTests > Task :deleteDependencyCache UP-TO-DATE > Task :executeOtherPHPUnitTests Execution according to PHPUnit XML configuration file './unittests/AllOtherPHPUnitTests.xml'... PHPUnit 6.5.8 by Sebastian Bergmann and contributors. ............................................. 45 / 45 (100%) Time: 1.62 minutes, Memory: 82.00MB OK (45 tests, 89 assertions) BUILD SUCCESSFUL in 1m 43s 2 actionable tasks: 1 executed, 1 up-to-date Resuming build at Thu Jun 28 08:29:44 CEST 2018 after Jenkins restart Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: ??? Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Resuming build at Thu Jun 28 08:32:27 CEST 2018 after Jenkins restart Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: ??? Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Resuming build at Thu Jun 28 08:34:59 CEST 2018 after Jenkins restart Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: ??? Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down Waiting to resume part of ACME-Pipeline 20180628-082349-rev163009: Jenkins is about to shut down [Pipeline] End of Pipeline java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677) at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787) at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:131) at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:112) at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:948) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1255) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1613) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149) at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135) at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53) at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425) at java.util.HashMap.readObject(HashMap.java:1409) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:309) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1644) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1752) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1665) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1292) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.lambda$readObject$0(RiverReader.java:250) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:237) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:250) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:782) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:775) at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) Caused: java.io.IOException: Failed to load build state at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:854) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:852) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:906) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE Is that a kind of timing issue: depending on which point during the sh step (that is running inside the Docker container) the Jenkins master restart is happening (of course I had used the "Manage Jenkins > Prepare for Shutdown" before)!?

          Jesse Glick added a comment -

          I cannot speculate on the cause.

          Jesse Glick added a comment - I cannot speculate on the cause.

          As commented above this issue has re-occurred => re-opening this issue...

          Reinhold Füreder added a comment - As commented above this issue has re-occurred => re-opening this issue...

          This problem has definitely re-occurred, because I accidentally restarted Jenkins master without waiting for pipelines to complete (cf. JENKINS-60434):

          ...
          [Pipeline] withEnv
          [Pipeline] {
          [Pipeline] withDockerRegistry
          [Pipeline] {
          [Pipeline] isUnix
          [Pipeline] sh
          07:34:43  + docker inspect -f . acme/mytool:13
          07:34:43  
          07:34:43  Error: No such object: acme/mytool:13
          [Pipeline] isUnix
          [Pipeline] sh
          07:34:43  + docker inspect -f . dockerregistry.acme.com/acme/mytool:13
          07:34:43  .
          [Pipeline] withDockerContainer
          07:34:43  Jenkins does not seem to be running inside a container
          07:34:43  $ docker run -t -d -u 10112:10005 --network=none -w /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS -v /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS:/var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS:rw,z -v /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS@tmp:/var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** dockerregistry.acme.com/acme/mytool:13 cat
          07:34:44  $ docker top 0a46f91e3a89720908a64eef2c8ac3688115a6472cb34161abec37b6bfc68e67 -eo pid,comm
          [Pipeline] {
          [Pipeline] lock
          07:34:44  Trying to acquire lock on [Label: MyTool, Quantity: 1]
          07:34:44  Lock acquired on [Label: MyTool, Quantity: 1]
          [Pipeline] {
          [Pipeline] pwd
          [Pipeline] sh
          07:34:44  + ... /opt/mytool/bin/...sh /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS ...
          ...
          07:34:48  Starting up MyTool  ...done.
          Resuming build at Wed Dec 18 07:35:36 CET 2019 after Jenkins restart
          Waiting to resume part of MyProject » MyProject-Pipeline 20191218-073417-rev4122d3ea: Finished waiting
          [Pipeline] End of Pipeline
          [lockable-resources] released lock on [MyTool#1, MyProject-Pipeline-mytool]
          java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image
          	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
          	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
          	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
          	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
          	at java.lang.Class.forName0(Native Method)
          	at java.lang.Class.forName(Class.java:348)
          	at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:123)
          	at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:104)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:1022)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1355)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220)
          	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220)
          	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
          	at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149)
          	at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135)
          	at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53)
          	at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307)
          	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
          	at java.util.HashMap.readObject(HashMap.java:1412)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadObject(JDKSpecific.java:179)
          	at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:212)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1746)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220)
          	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
          	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:205)
          	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
          	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.lambda$readObject$0(RiverReader.java:250)
          	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:237)
          	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:237)
          	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:250)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:783)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:776)
          	at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
          	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
          	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
          	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
          	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
          	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
          	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
          	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48)
          	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          Caused: java.io.IOException: Failed to load build state
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:855)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:853)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:907)
          	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE
          

          Please note that the Docker container was still running and I therefore had to stop it...

          $ docker ps -a
          CONTAINER ID        IMAGE                                          COMMAND                  CREATED             STATUS              PORTS                    NAMES
          0a46f91e3a89        dockerregistry.acme.com/acme/mytool:13   "cat"                    17 minutes ago      Up 17 minutes                                nifty_heisenberg
          ...
          

          Reinhold Füreder added a comment - This problem has definitely re-occurred, because I accidentally restarted Jenkins master without waiting for pipelines to complete (cf. JENKINS-60434 ): ... [Pipeline] withEnv [Pipeline] { [Pipeline] withDockerRegistry [Pipeline] { [Pipeline] isUnix [Pipeline] sh 07:34:43 + docker inspect -f . acme/mytool:13 07:34:43 07:34:43 Error: No such object: acme/mytool:13 [Pipeline] isUnix [Pipeline] sh 07:34:43 + docker inspect -f . dockerregistry.acme.com/acme/mytool:13 07:34:43 . [Pipeline] withDockerContainer 07:34:43 Jenkins does not seem to be running inside a container 07:34:43 $ docker run -t -d -u 10112:10005 --network=none -w /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS -v /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS:/var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS:rw,z -v /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS@tmp:/var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** dockerregistry.acme.com/acme/mytool:13 cat 07:34:44 $ docker top 0a46f91e3a89720908a64eef2c8ac3688115a6472cb34161abec37b6bfc68e67 -eo pid,comm [Pipeline] { [Pipeline] lock 07:34:44 Trying to acquire lock on [Label: MyTool, Quantity: 1] 07:34:44 Lock acquired on [Label: MyTool, Quantity: 1] [Pipeline] { [Pipeline] pwd [Pipeline] sh 07:34:44 + ... /opt/mytool/bin/...sh /var/lib/jenkins/workspace/MyProject/MyProject-Pipeline-WS ... ... 07:34:48 Starting up MyTool ...done. Resuming build at Wed Dec 18 07:35:36 CET 2019 after Jenkins restart Waiting to resume part of MyProject » MyProject-Pipeline 20191218-073417-rev4122d3ea: Finished waiting [Pipeline] End of Pipeline [lockable-resources] released lock on [MyTool#1, MyProject-Pipeline-mytool] java.lang.ClassNotFoundException: org.jenkinsci.plugins.docker.workflow.Docker$Image at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677) at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787) at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:123) at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:104) at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:1022) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1355) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149) at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135) at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53) at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425) at java.util.HashMap.readObject(HashMap.java:1412) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadObject(JDKSpecific.java:179) at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:212) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1746) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:205) at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.lambda$readObject$0(RiverReader.java:250) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:237) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:237) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:250) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:783) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:776) at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) Caused: java.io.IOException: Failed to load build state at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:855) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:853) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:907) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE Please note that the Docker container was still running and I therefore had to stop it... $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0a46f91e3a89 dockerregistry.acme.com/acme/mytool:13 "cat" 17 minutes ago Up 17 minutes nifty_heisenberg ...

          Jesse Glick added a comment -

          No idea as to the root cause. Again the workaround would be simple: avoid the DSL and use withDockerContainer directly. (Or, better yet, avoid the whole plugin and run docker CLI commands from sh.)

          Jesse Glick added a comment - No idea as to the root cause. Again the workaround would be simple: avoid the DSL and use withDockerContainer directly. (Or, better yet, avoid the whole plugin and run docker CLI commands from sh .)

            Unassigned Unassigned
            reinholdfuereder Reinhold Füreder
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: