Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50407

NPE from SandboxContinuable.run0 after restart

      In this build I observed

      ...
      Running in Durability level: PERFORMANCE_OPTIMIZED
      ...
      Resuming build at ... after Jenkins restart
      [Pipeline] End of Pipeline
      
      GitHub has been notified of this commit's build result
      
      java.lang.NullPointerException
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:43)
      	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:331)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:82)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:243)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:231)
      	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
      	at ...
      Finished: FAILURE
      

      The flow graph is also missing. Not sure what happened here.

          [JENKINS-50407] NPE from SandboxContinuable.run0 after restart

          Jesse Glick added a comment -

          svanoort claimed a possible link to a couple of other bugs under investigation.

          Jesse Glick added a comment - svanoort claimed a possible link to a couple of other bugs under investigation.

          Sam Van Oort added a comment -

          I'll upgrade that from "possible" to guaranteed, knowing that the flow graph is missing too. That's a pipeline that was not cleanly persisted in performance optimized mode – and yet it is trying to resume (when it should be forbidden).

          I'm armor-plating all the resumption & completion-state logic just to try to make this a bit more failsafe.

          Sam Van Oort added a comment - I'll upgrade that from "possible" to guaranteed, knowing that the flow graph is missing too. That's a pipeline that was not cleanly persisted in performance optimized mode – and yet it is trying to resume (when it should be forbidden). I'm armor-plating all the resumption & completion-state logic just to try to make this a bit more failsafe.

          Jesse Glick added a comment -

          In Review perhaps?

          Jesse Glick added a comment - In Review perhaps?

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java
          src/test/java/org/jenkinsci/plugins/workflow/job/WorkflowRunRestartTest.java
          src/test/java/org/jenkinsci/plugins/workflow/job/WorkflowRunTest.java
          http://jenkins-ci.org/commit/workflow-job-plugin/f0c26058f31d4f159a82a3cace52935e93f20701
          Log:
          Merge pull request #93 from svanoort/fix-resume-issues

          Fix resume issues JENKINS-49686 and JENKINS-50199 and JENKINS-50407

          Compare: https://github.com/jenkinsci/workflow-job-plugin/compare/e11cea623f61...f0c26058f31d

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java src/test/java/org/jenkinsci/plugins/workflow/job/WorkflowRunRestartTest.java src/test/java/org/jenkinsci/plugins/workflow/job/WorkflowRunTest.java http://jenkins-ci.org/commit/workflow-job-plugin/f0c26058f31d4f159a82a3cace52935e93f20701 Log: Merge pull request #93 from svanoort/fix-resume-issues Fix resume issues JENKINS-49686 and JENKINS-50199 and JENKINS-50407 Compare: https://github.com/jenkinsci/workflow-job-plugin/compare/e11cea623f61...f0c26058f31d

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/5d08569babf0249ddd3b0723bcf5671df1e48ac7
          Log:
          JENKINS-50407 Trying to diagnose an NPE.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java http://jenkins-ci.org/commit/workflow-cps-plugin/5d08569babf0249ddd3b0723bcf5671df1e48ac7 Log: JENKINS-50407 Trying to diagnose an NPE.

          Sam Van Oort added a comment -

          Resolved as of workflow-cps 2.47 and workflow-job 2.18

          Sam Van Oort added a comment - Resolved as of workflow-cps 2.47 and workflow-job 2.18

          Ulli Hafner added a comment -

          I'm not sure if this is interesting for this issue: I got the following exception in my CI build:

          java.lang.IllegalStateException: JENKINS-50407: no loaded shell in CpsFlowExecution[Owner[Plugins/warnings-ng-plugin/PR-551/5:Plugins/warnings-ng-plugin/PR-551 #5]]
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:35)
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
          17:27:21  	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
          17:27:21  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          17:27:21  	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          17:27:21  	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          17:27:21  	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
          17:27:21  	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          17:27:21  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          17:27:21  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          17:27:21  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          17:27:21  	at java.base/java.lang.Thread.run(Thread.java:834)
          

          Ulli Hafner added a comment - I'm not sure if this is interesting for this issue: I got the following exception in my CI build : java.lang.IllegalStateException: JENKINS-50407: no loaded shell in CpsFlowExecution[Owner[Plugins/warnings-ng-plugin/PR-551/5:Plugins/warnings-ng-plugin/PR-551 #5]] 17:27:21 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:35) 17:27:21 at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185) 17:27:21 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400) 17:27:21 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96) 17:27:21 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312) 17:27:21 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276) 17:27:21 at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67) 17:27:21 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 17:27:21 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) 17:27:21 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) 17:27:21 at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) 17:27:21 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) 17:27:21 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 17:27:21 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 17:27:21 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 17:27:21 at java.base/java.lang.Thread.run(Thread.java:834)

          Liam Newman added a comment -

          drulli The next time you see this issue, please provide some of the logs before and after the exception for context.  Thanks!

           

          Liam Newman added a comment - drulli The next time you see this issue, please provide some of the logs before and after the exception for context.  Thanks!  

          Ulli Hafner added a comment -

          Yes I will try. The exception occurred on the Jenkins CI instance where I do not have access to the logs...

          Ulli Hafner added a comment - Yes I will try. The exception occurred on the Jenkins CI instance where I do not have access to the logs...

          Clerc Mathias added a comment -

           

          We had a similar issue in one of our builds:

          Resuming build at Tue Aug 25 08:30:32 JST 2020 after Jenkins restart
          [Pipeline] End of Pipeline
          [lockable-resources] released lock on [benchmark]
          java.lang.IllegalStateException: JENKINS-50407: no loaded shell in CpsFlowExecution[Owner[Benchmark/master/20:Benchmark/master #20]]
          	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:35)
          	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
          	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
          	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
          	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
          	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          	at java.base/java.lang.Thread.run(Thread.java:834)
          Finished: FAILURE
          

          Some more information on the build:

           

          • Jenkins 2.235.5
          • Scripted pipeline
          • Running on a single node
          • The pipeline contains disableResume, which makes it strange it tried to resume at all:
          options {
              disableConcurrentBuilds()
              disableResume()
              [...]
          }
          

          Before that the logs are just the automated benchmarks running normally and there is nothing coming afterwards.
           

          Clerc Mathias added a comment -   We had a similar issue in one of our builds: Resuming build at Tue Aug 25 08:30:32 JST 2020 after Jenkins restart [Pipeline] End of Pipeline [lockable-resources] released lock on [benchmark] java.lang.IllegalStateException: JENKINS-50407: no loaded shell in CpsFlowExecution[Owner[Benchmark/master/20:Benchmark/master #20]] at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:35) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang. Thread .run( Thread .java:834) Finished: FAILURE Some more information on the build:   Jenkins 2.235.5 Scripted pipeline Running on a single node The pipeline contains disableResume, which makes it strange it tried to resume at all: options { disableConcurrentBuilds() disableResume() [...] } Before that the logs are just the automated benchmarks running normally and there is nothing coming afterwards.  

            svanoort Sam Van Oort
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: