-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
Jenkins 2.48
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
-
Powered by SuggestiMate
Mar 07, 2017 12:39:15 PM SEVERE hudson.remoting.ExportTable unexportByOid
Trying to unexport an object that's already unexported
java.lang.Exception: Object appears to be deallocated at lease before Tue Mar 07 12:33:23 EET 2017
at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:445)
Caused: java.util.concurrent.ExecutionException: Invalid object ID 92 iota=55
at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:449)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:487)
at hudson.remoting.Channel.unexport(Channel.java:691)
at hudson.remoting.Channel.unexport(Channel.java:680)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$1.handle(Channel.java:527)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83)
- is duplicated by
-
JENKINS-50498 Invalid Object ID in remoting module related to Proxy methods
-
- Resolved
-
- is related to
-
JENKINS-23271 Intermittent Invalid Object ID in remoting module
-
- Resolved
-
[JENKINS-42533] Trying to unexport an object that's already unexported
- Slave jar is download always from master so it was same version as master: wget -O /mnt/jenkins/slave.jar http://jenkins.company.domain/jnlpJars/slave.jar
Here are snipets from logs (I don't know how to delivery logs for you and not for everybody):
Mar 06, 2017 10:38:08 AM hudson.remoting.ExportTable unexportByOid SEVERE: Trying to unexport an object that's already unexported java.util.concurrent.ExecutionException: Invalid object ID 27 iota=60 at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:449) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:487) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$1.handle(Channel.java:527) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83) Caused by: java.lang.Exception: Object was recently deallocated #27 (ref.0) : object=null type=hudson.CloseProofOutputStream interfaces=[java.io.OutputStream] Created at Mon Mar 06 10:35:07 EET 2017 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:106) at hudson.remoting.ExportTable.export(ExportTable.java:338) at hudson.remoting.Channel.internalExport(Channel.java:663) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor226.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:178) at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.remoting.UserRequest._serialize(UserRequest.java:190) at hudson.remoting.UserRequest.serialize(UserRequest.java:199) at hudson.remoting.UserRequest.<init>(UserRequest.java:64) at hudson.remoting.Channel.call(Channel.java:828) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:959) at hudson.Launcher$ProcStarter.start(Launcher.java:384) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:405) Released at Mon Mar 06 10:37:20 EET 2017 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:155) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:492) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:344) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19) at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: Command ProxyOutputStream.Unexport(27) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330) at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:442) ... 6 more Caused by: Released at Mon Mar 06 10:37:20 EET 2017 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:155) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:492) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:344) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19) at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: Command ProxyOutputStream.Unexport(27) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330) at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) Mar 06, 2017 10:38:08 AM hudson.remoting.ExportTable unexportByOid SEVERE: 2nd unexport attempt is here Command hudson.remoting.UnexportCommand@3a04c0cb created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) Mar 06, 2017 10:38:32 AM hudson.remoting.ExportTable unexportByOid SEVERE: Trying to unexport an object that's already unexported java.util.concurrent.ExecutionException: Invalid object ID 27 iota=60 at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:449) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:487) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$1.handle(Channel.java:527) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83) Caused by: java.lang.Exception: Object was recently deallocated #27 (ref.0) : object=null type=hudson.CloseProofOutputStream interfaces=[java.io.OutputStream] Created at Mon Mar 06 10:35:07 EET 2017 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:106) at hudson.remoting.ExportTable.export(ExportTable.java:338) at hudson.remoting.Channel.internalExport(Channel.java:663) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor226.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:178) at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.remoting.UserRequest._serialize(UserRequest.java:190) at hudson.remoting.UserRequest.serialize(UserRequest.java:199) at hudson.remoting.UserRequest.<init>(UserRequest.java:64) at hudson.remoting.Channel.call(Channel.java:828) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:959) at hudson.Launcher$ProcStarter.start(Launcher.java:384) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:405) Released at Mon Mar 06 10:37:20 EET 2017 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:155) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:492) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:344) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19) at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: Command ProxyOutputStream.Unexport(27) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330) at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:442) ... 6 more Caused by: Released at Mon Mar 06 10:37:20 EET 2017 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:155) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:492) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:344) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19) at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: Command ProxyOutputStream.Unexport(27) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330) at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) Mar 06, 2017 10:38:32 AM hudson.remoting.ExportTable unexportByOid SEVERE: 2nd unexport attempt is here Command hudson.remoting.UnexportCommand@646b91f9 created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
Don't know which slave caused the problem or where to get old agent log. We are using ssh to launch agent and the jenkins is restarted already because it stuck for some reason. Problem for jenkins started after 2.48. Now we are using 2.49 version.
So it is related to the fix of JENKINS-22853 in https://github.com/jenkinsci/remoting/pull/97/files
There I have fixed the ProxyOutputStream#EOF command, but not ProxyOutputStream#Unexport(). Maybe makes sense to fix it as well, will investigate
I'm seeing this with the Parameterized Trigger Plugin. See https://issues.jenkins-ci.org/browse/JENKINS-43304
The logs show:
Apr 03, 2017 10:56:07 PM hudson.remoting.ExportTable unexportByOid SEVERE: Trying to unexport an object that's already unexported java.util.concurrent.ExecutionException: Invalid object ID 347 iota=3443 at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:449) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:487) at hudson.remoting.Channel.unexport(Channel.java:691) at hudson.remoting.Channel.unexport(Channel.java:680) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$1.handle(Channel.java:527) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83) Caused by: java.lang.Exception: Object appears to be deallocated at lease before Mon Apr 03 22:13:31 CEST 2017 at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:445) ... 6 more Apr 03, 2017 10:56:07 PM hudson.remoting.ExportTable unexportByOid SEVERE: 2nd unexport attempt is here Command hudson.remoting.UnexportCommand@1aabb68c created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221) at java.lang.System$2.invokeFinalize(Unknown Source) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
Unfortunately I have no capacity to work on Remoting in medium term, so I will unassign it and let others to take it. If somebody is interested to submit a pull request, I will be happy to help to get it reviewed and released.
Released in the latest 3.26 version of Remoting. Merged to go into the next Jenkins release.
Please provide...
Likely it's just another occurrence of
JENKINS-23271