We've seen a rare ClassCastException that appears to be an object serialization stream issue:
Jul 26, 2013 12:19:00 PM hudson.maven.ProcessCache$MavenProcess call FINE: Maven JVM terminated unexpectedly with exit code 143 hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41) at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34) at hudson.remoting.Request.call(Request.java:174) at hudson.remoting.Channel.call(Channel.java:718) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:156) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:757) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:594) at hudson.model.Run.execute(Run.java:1575) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:237) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination at hudson.remoting.Request.abort(Request.java:299) at hudson.remoting.Channel.terminate(Channel.java:778) at hudson.slaves.Channels$1.terminate(Channels.java:77) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:72) Caused by: java.io.IOException: Unexpected reader termination ... 1 more Caused by: java.lang.ClassCastException: cannot assign instance of hudson.remoting.ProxyOutputStream$Chunk to field java.lang.Throwable.suppressedExceptions of type java.util.List in instance of hudson.remoting.Command$Source at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(Unknown Source) at java.io.ObjectStreamClass.setObjFieldValues(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.defaultReadObject(Unknown Source) at java.lang.Throwable.readObject(Throwable.java:913) at sun.reflect.GeneratedMethodAccessor1626.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.defaultReadObject(Unknown Source) at java.lang.Throwable.readObject(Throwable.java:913) at sun.reflect.GeneratedMethodAccessor1626.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.defaultReadObject(Unknown Source) at java.lang.Throwable.readObject(Throwable.java:913) at sun.reflect.GeneratedMethodAccessor1626.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.defaultReadObject(Unknown Source) at java.lang.Throwable.readObject(Throwable.java:913) at sun.reflect.GeneratedMethodAccessor1626.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Command.readFrom(Command.java:92) at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:72) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
Jul 23, 2013 2:24:27 PM hudson.maven.ProcessCache$MavenProcess call FINE: Maven JVM terminated unexpectedly with exit code 143 hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41) at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34) at hudson.remoting.Request.call(Request.java:174) at hudson.remoting.Channel.call(Channel.java:718) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:156) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:757) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:594) at hudson.model.Run.execute(Run.java:1575) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:237) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected reader termination at hudson.remoting.Request.abort(Request.java:299) at hudson.remoting.Channel.terminate(Channel.java:778) at hudson.slaves.Channels$1.terminate(Channels.java:77) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:72) Caused by: java.io.IOException: Unexpected reader termination ... 1 more Caused by: java.lang.ClassCastException: cannot assign instance of hudson.remoting.UserRequest to field hudson.remoting.Command.createdAt of type java.lang.Exception in instance of hudson.remoting.UnexportCommand at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(Unknown Source) at java.io.ObjectStreamClass.setObjFieldValues(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Command.readFrom(Command.java:92) at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:72) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
- is duplicated by
-
JENKINS-19349 Exception during build
-
- Closed
-
I'm reading the protocol spec, and I'm really having hard time coming up with a plausible scenario that would cause this problem.
Within the paired readObject and writeObject, it's easy to cause this. But it's very hard for an object written in Foo.writeObject() to be read by Bar.readObject() because of TC_ENDBLOCKDATA enforcement at the boundary. Every time the application code gets to write stuff to a stream, the spec puts TC_ENDBLOCKDATA marker at the end to catch errors.
The other variable length places (such as how it reads field values for object) are dictated by the class descriptor that the writer sends, so the reader can't get out of sync there.
Even the arbitrary byte[] data written gets enveloped into TC_BLOCKDATA, so you can't make a reader get out of sync by reading more bytes from read(byte[]) than you wrote. Ditto for primitive values.
So unless the sequence of bytes are tampered after it left ObjectOutputStream and before read by ObjectInputStream, the framing logic feels ironcrad. And such a stream corruption normally occurs a boundary between data burst, and not in the middle like this.
I'm filing this ticket here to see if anyone else saw the same problem.