-
Bug
-
Resolution: Fixed
-
Minor
-
-
Pipeline - April 2018
From the logs of ci.jenkins.io
Dec 11, 2017 7:16:02 PM org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService reportProblem WARNING: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[Infra/plugin-site-api/generate-data/14477:Infra/plugin-site-api/generate-data #14477]] java.lang.NullPointerException at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:342) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:243) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:231) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Dec 11, 2017 7:16:02 PM org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService reportProblem WARNING: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[Infra/plugin-site-api/generate-data/14476:Infra/plugin-site-api/generate-data #14476]] java.lang.NullPointerException at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:342) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:243) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:231) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Dec 11, 2017 7:16:02 PM org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService reportProblem WARNING: Unexpected exception in CPS VM thread: CpsFlowExecution[Owner[Infra/plugin-site-api/generate-data/14475:Infra/plugin-site-api/generate-data #14475]] java.lang.NullPointerException at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:342) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:243) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:231) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
[JENKINS-48491] NPE in CpsThreadGroup.run & IOOBE in CpsThreadGroup.readResolve
| Component/s | New: workflow-cps-plugin [ 21713 ] | |
| Component/s | Original: pipeline [ 21692 ] |
| Sprint | New: Pipeline - December [ 446 ] |
Looking at the specific runs that are referenced here, I see this in the last FlowNode XML it generated.
<error class="java.io.IOException"> <detailMessage>Failed to load build state</detailMessage> <cause class="java.lang.IndexOutOfBoundsException"> <detailMessage>Index: 0, Size: 0</detailMessage> <cause class="org.jboss.marshalling.TraceInformation" plugin="workflow-support@2.16"> <stackTrace/> <suppressedExceptions class="java.util.Collections$UnmodifiableRandomAccessList" resolves-to="java.util.Collections$UnmodifiableList"> <c class="list"/> <list reference="../c"/> </suppressedExceptions> <info class="org.jboss.marshalling.TraceInformation$IncompleteObjectInfo"> <targetClassName>org.jenkinsci.plugins.workflow.cps.CpsThreadGroup</targetClassName> </info> </cause> <stackTrace> <trace>java.util.ArrayList.rangeCheck(ArrayList.java:653)</trace> <trace>java.util.ArrayList.get(ArrayList.java:429)</trace> <trace>org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.readResolve(CpsThreadGroup.java:157)</trace> <trace>sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)</trace> <trace>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</trace> <trace>java.lang.reflect.Method.invoke(Method.java:498)</trace> <trace>org.jboss.marshalling.reflect.SerializableClass.callReadResolve(SerializableClass.java:415)</trace> <trace>org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1293)</trace> <trace>org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:276)</trace> <trace>org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)</trace> <trace>org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)</trace> <trace>org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:627)</trace> <trace>org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:620)</trace> <trace>org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150)</trace> <trace>com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)</trace> <trace>com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)</trace> <trace>com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)</trace> <trace>com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)</trace> <trace>org.jenkinsci.plugins.workflow.support.concurrent.Futures.addCallback(Futures.java:160)</trace> <trace>org.jenkinsci.plugins.workflow.support.concurrent.Futures.addCallback(Futures.java:90)</trace> <trace>org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.loadProgramAsync(CpsFlowExecution.java:617)</trace> <trace>org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.onLoad(CpsFlowExecution.java:589)</trace> <trace>org.jenkinsci.plugins.workflow.job.WorkflowRun.onLoad(WorkflowRun.java:613)</trace> <trace>hudson.model.RunMap.retrieve(RunMap.java:225)</trace> <trace>hudson.model.RunMap.retrieve(RunMap.java:57)</trace> ...
The wonderfully weird thing about that is that right before line 157, where it does call scripts.get(0), is if (!scripts.isEmpty()). Whaaaat.
Note that all the builds with this problem seem to be almost four months old and attempted to resume a few weeks ago (according to the timestamp on the 2.xml and 3.xml files). They still have program.dat files, which date to four days ago, and their log files date to the last restart of ci.jenkins.io.
Obviously, the logspam can be eliminated by just deleting the weirdo builds, but there's at least a couple levels of weirdness here - the IndexOutOfBoundsException in CpsThreadGroup#readResolve, and above that, the NullPointerException in CpsThreadGroup#run. The NPE is probably easily worked around as jglick laid out, but I'm legitimately curious as to how the IndexOutOfBoundsException can actually happen, given that we just verified that the list in question isn't empty. Good times?
A little more detail - I cut off the stack trace from the FlowNode before I should have. It follows with...
<trace>hudson.model.RunMap.retrieve(RunMap.java:225)</trace>
<trace>hudson.model.RunMap.retrieve(RunMap.java:57)</trace>
<trace>jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:500)</trace>
<trace>jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:482)</trace>
<trace>jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:380)</trace>
<trace>jenkins.model.lazy.LazyBuildMixIn.getBuildByNumber(LazyBuildMixIn.java:231)</trace>
<trace>org.jenkinsci.plugins.workflow.job.WorkflowJob.getBuildByNumber(WorkflowJob.java:228)</trace>
<trace>io.jenkins.blueocean.rest.impl.pipeline.PipelineRunImpl.getCommitUrl(PipelineRunImpl.java:231)</trace>
<trace>io.jenkins.blueocean.commons.stapler.export.MethodProperty.getValue(MethodProperty.java:72)</trace>
<trace>io.jenkins.blueocean.commons.stapler.export.ExportInterceptor$1.getValue(ExportInterceptor.java:43)</trace>
<trace>io.jenkins.blueocean.commons.stapler.Export$BlueOceanExportInterceptor.getValue(Export.java:167)</trace>
<trace>io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:136)</trace>
<trace>io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)</trace>
<trace>io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:199)</trace>
<trace>io.jenkins.blueocean.commons.stapler.Export.writeOne(Export.java:148)</trace>
<trace>io.jenkins.blueocean.commons.stapler.Export.serveExposedBean(Export.java:139)</trace>
<trace>io.jenkins.blueocean.commons.stapler.Export.doJson(Export.java:79)</trace>
<trace>io.jenkins.blueocean.commons.stapler.TreeResponse$Processor$1.generateResponse(TreeResponse.java:48)</trace>
And then a bunch more stapler stuff. Smells to me like Blue Ocean is trying to do something with the run at the same time it's loading, and things are colliding badly.
Moving to Blue Ocean since this is being caused by something Blue Ocean is doing.
| Component/s | New: blueocean-plugin [ 21481 ] | |
| Component/s | Original: workflow-cps-plugin [ 21713 ] | |
| Assignee | Original: Andrew Bayer [ abayer ] |
| Summary | Original: NPE in Pipeline: Unexpected exception in CPS VM thread: CpsFlowExecutio | New: NPE in CpsThreadGroup.run & IOOBE in |
Somehow an exception is being thrown and then there is no active thread corresponding to it. No clue how this could happen, but anyway this is probably a build-ending event so if something here is unexpectedly null, log what it is and skip the ErrorAction for now. I do not really follow the logic here and how it interacts with exception handling in parallel blocks.