-
Bug
-
Resolution: Fixed
-
Major
-
Jenkins 2.155
Workflow Support 3.0-java11-alpha-1
-
-
workflow-support 3.0
While testing config-file-provider plugin on PCT, setting the dependency to workflow-support 3.0-java11-alpha-1, I discovered an NPE on the marshalling
java.lang.NullPointerException at com.cloudbees.groovy.cps.SerializableScript.readObject(SerializableScript.java:31) at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadObject(JDKSpecific.java:100) at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:212) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1742) at
Full log:
hudson.remoting.ProxyException: an exception which occurred:hudson.remoting.ProxyException: an exception which occurred: in object of type WorkflowScript in field groovy.lang.Closure.delegate in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@73fa5905 in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2 in object of type java.util.HashMap in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.closures in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@63ea2c01 in object of type org.jenkinsci.plugins.workflow.cps.CpsThreadGroupCaused: hudson.remoting.ProxyException: java.lang.NullPointerException at com.cloudbees.groovy.cps.SerializableScript.readObject(SerializableScript.java:31) at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadObject(JDKSpecific.java:100) at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:212) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1742) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1711) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1711) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1391) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1849) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1763) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1711) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1711) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1391) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149) at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135) at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53) at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307) at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:424) at java.base/java.util.HashMap.readObject(HashMap.java:1460) at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadObject(JDKSpecific.java:100) at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:212) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1742) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1391) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220) at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1849) at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1763) at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1391) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:205) at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.lambda$readObject$0(RiverReader.java:250) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:237) at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:250) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:627) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:620) at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40) at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155) at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149) at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)Caused: hudson.remoting.ProxyException: java.io.IOException: Failed to load build state at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:699) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:697) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:746) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)
- blocks
-
JENKINS-52187 Pipeline: Support. Integrate JBoss Remoting patches in the common release
-
- Resolved
-
- is duplicated by
-
-
- Closed
-
- links to
[JENKINS-55174] Pipeline: Support. NPE in build state marshalling
Sam Van Oort
added a comment - oleg_nenashev Yup, that's the very same bugbear that's had me pulling out my hair for a while now. Right now I'm working on an alternate way of debugging it.
Sam Van Oort
added a comment - oleg_nenashev Yup, that's the very same bugbear that's had me pulling out my hair for a while now. Right now I'm working on an alternate way of debugging it. Sam Van Oort
added a comment - Writing up results of pair-investigation with dnusbaum today and summarizing some previous findings
0. I'd previously established that the culprit here is a null "binding" field in the WorkflowScript when deserializing, and had not been able to determine how the setting of that field differs between JBoss Marshalling versions
1. This issue appears to occur with even very trivial Pipelines when trying to read the program.dat – see: https://github.com/svanoort/workflow-support-plugin/commit/60dff8f944407a46e70f1fcfc3843cf7da29d974#diff-5592588df61e52ede2b60dbc65f7009bR56
2. Some subtle differences in the serialization format between jboss-marshalling 1.4.12-jenkins-3 and 2.0.5.Final (I'd bisected the issue and confirmed it occurs as early as version 2.0.0 previously) – see attached program.dat files – but analyzing them briefly in a hex editor and via 'strings' does not show substantial differences. I.E. it's not like things simply aren't being serialized, the format is just modified to be a bit more efficient.
3. Stack traces when trying to read the SerializableScript are attached – note that the stacks seem more or less identical, aside froma switch for Java version changes, and different line numbers due to source changes in JBoss marshalling
4. Nothing obvious in the JBoss marshalling JIRA that would reflect this bug
5. Nothing obvious jumping out in the commits for JBoss marshalling that have changed https://github.com/jboss-remoting/jboss-marshalling/compare/1.4.12.Final...2.0.0.Final
See data dumps:
jboss-NEW-marshaller-unmarshal-thread.txt
jboss-old-marshaller-unmarshal-thread.txt test-java8-LEGACY-marshaller-program.dat test-java8-new-marshaller-program.dat
Sam Van Oort
added a comment - Writing up results of pair-investigation with dnusbaum today and summarizing some previous findings
0. I'd previously established that the culprit here is a null "binding" field in the WorkflowScript when deserializing, and had not been able to determine how the setting of that field differs between JBoss Marshalling versions
1. This issue appears to occur with even very trivial Pipelines when trying to read the program.dat – see: https://github.com/svanoort/workflow-support-plugin/commit/60dff8f944407a46e70f1fcfc3843cf7da29d974#diff-5592588df61e52ede2b60dbc65f7009bR56
2. Some subtle differences in the serialization format between jboss-marshalling 1.4.12-jenkins-3 and 2.0.5.Final (I'd bisected the issue and confirmed it occurs as early as version 2.0.0 previously) – see attached program.dat files – but analyzing them briefly in a hex editor and via 'strings' does not show substantial differences. I.E. it's not like things simply aren't being serialized, the format is just modified to be a bit more efficient.
3. Stack traces when trying to read the SerializableScript are attached – note that the stacks seem more or less identical, aside froma switch for Java version changes, and different line numbers due to source changes in JBoss marshalling
4. Nothing obvious in the JBoss marshalling JIRA that would reflect this bug
5. Nothing obvious jumping out in the commits for JBoss marshalling that have changed https://github.com/jboss-remoting/jboss-marshalling/compare/1.4.12.Final...2.0.0.Final
See data dumps:
jboss-NEW-marshaller-unmarshal-thread.txt jboss-old-marshaller-unmarshal-thread.txt test-java8-LEGACY-marshaller-program.dat test-java8-new-marshaller-program.dat Sam Van Oort
added a comment - oleg_nenashev I've attached a bunch of info from investigation above, but am running out of investigative ideas here aside from manually building and bisecting all the JBoss marshalling commits here, or tracing line-by-line through all the mutations of the SerialializableScript in the marshaller (I've done quite a bit of poking with the debugger & breakpoints here already).
Would love to know if you have any ideas for how to attack this particular beast.
Sam Van Oort
added a comment - oleg_nenashev I've attached a bunch of info from investigation above, but am running out of investigative ideas here aside from manually building and bisecting all the JBoss marshalling commits here, or tracing line-by-line through all the mutations of the SerialializableScript in the marshaller (I've done quite a bit of poking with the debugger & breakpoints here already).
Would love to know if you have any ideas for how to attack this particular beast. Sam Van Oort
added a comment - My suspicion is that one of the following is the cause based on the evidence above:
1. Unlikely: a very subtle change in the serialized format causes a net-new bug in JBoss Marshalling that prevents correct serialization.
2. Likely: Some subtlety in how readReplace methods are being handled (perhaps with regards to customizations in workflow or Groovy) – or especially with regards to order of operations here. This is supported by limited differences in the serialized output.
Sam Van Oort
added a comment - My suspicion is that one of the following is the cause based on the evidence above:
1. Unlikely: a very subtle change in the serialized format causes a net-new bug in JBoss Marshalling that prevents correct serialization.
2. Likely: Some subtlety in how readReplace methods are being handled (perhaps with regards to customizations in workflow or Groovy) – or especially with regards to order of operations here. This is supported by limited differences in the serialized output. At the yesterday's Platform SIG meeting we agreed to reach out to JBoss Marshalling maintainers. I also do not have much idea what could be the root cause.
I think our issue is a duplicate of JBMAR-221, which was fixed in JBoss Marshalling 2.0.6.Final. I've opened https://github.com/jenkinsci/workflow-support-plugin/pull/86 against the Java 11 branch to update the library and unignore the previously-failing tests.
I believe this is the same issue as svanoort trying to fix for the GA release. It was reported for tests in https://github.com/jenkinsci/workflow-support-plugin/pull/84