-
Bug
-
Resolution: Fixed
-
Blocker
-
Powered by SuggestiMate
Just upgraded to 1.561, and I'm happy to see JENKINS-20769 fixed, but now I'm seeing these instead:
SEVERE: Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 6 iota=36 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: java.lang.Exception: Object was recently deallocated #6 (ref.0) : hudson.remoting.FastPipedOutputStream Created at Fri May 02 09:50:35 PDT 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.Pipe.writeObject(Pipe.java:158) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.callAsync(Channel.java:766) at hudson.FilePath.read(FilePath.java:1640) at hudson.FilePath.readToString(FilePath.java:1662) at hudson.tools.JDKInstaller.performInstallation(JDKInstaller.java:126) at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:61) at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:107) at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:204) at hudson.model.JDK.forNode(JDK.java:126) at org.jenkinsci.plugins.envinject.service.EnvInjectVariableGetter.getBuildVariables(EnvInjectVariableGetter.java:79) at org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironmentWithoutJobPropertyObject(EnvInjectListener.java:231) at org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:46) at hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:570) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:476) at hudson.model.Run.execute(Run.java:1709) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Released at Fri May 02 09:50:35 PDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$EOF$1.run(ProxyOutputStream.java:384) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 5 more Caused by: Released at Fri May 02 09:50:35 PDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$EOF$1.run(ProxyOutputStream.java:384) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)
- is related to
-
JENKINS-23271 Intermittent Invalid Object ID in remoting module
-
- Resolved
-
-
JENKINS-18781 Configurable channel timeout for slaves
-
- Open
-
- links to
[JENKINS-22853] SEVERE: Trying to unexport an object that's already unexported
I just attempted to upgrade to 1.564, but had to revert again - JENKINS-23120
I just completed upgrading to 1.564 and have captured the following logging information. It would appear that our environment is reporting these following stack traces repeatedly. I will continue to monitor the environment. We did upgrade the master node to java 8 as well as apart of this upgrade.
May 25, 2014 11:00:07 AM SEVERE hudson.remoting.ExportTable unexportByOid Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 5 iota=171 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.Exception: Object was recently deallocated #5 (ref.0) : hudson.CloseProofOutputStream Created at Sun May 25 10:24:01 CDT 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:161) at sun.reflect.GeneratedMethodAccessor141.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.call(Channel.java:738) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:888) at hudson.Launcher$ProcStarter.start(Launcher.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:362) at hudson.tools.AbstractCommandInstaller.performInstallation(AbstractCommandInstaller.java:77) at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:61) at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:107) at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:204) at hudson.plugins.git.GitTool.forNode(GitTool.java:79) at hudson.plugins.git.GitSCM.getGitExe(GitSCM.java:679) at hudson.plugins.git.GitSCM.getGitExe(GitSCM.java:658) at hudson.plugins.git.GitSCM.createClient(GitSCM.java:563) at hudson.plugins.git.GitSCM.createClient(GitSCM.java:558) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:874) at hudson.model.AbstractProject.checkout(AbstractProject.java:1251) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513) at hudson.model.Run.execute(Run.java:1706) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Released at Sun May 25 10:24:05 CDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 13 more Caused by: Released at Sun May 25 10:24:05 CDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
May 25, 2014 11:00:07 AM SEVERE hudson.remoting.ExportTable unexportByOid Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 53 iota=171 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.Exception: Object was recently deallocated #53 (ref.0) : hudson.CloseProofOutputStream Created at Sun May 25 10:34:30 CDT 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:161) at sun.reflect.GeneratedMethodAccessor141.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.call(Channel.java:738) at hudson.FilePath.act(FilePath.java:909) at hudson.FilePath.act(FilePath.java:893) at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:66) at hudson.plugins.git.GitSCM.createClient(GitSCM.java:566) at hudson.plugins.git.GitSCM.createClient(GitSCM.java:558) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:874) at hudson.model.AbstractProject.checkout(AbstractProject.java:1251) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513) at hudson.model.Run.execute(Run.java:1706) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Released at Sun May 25 10:50:19 CDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 13 more Caused by: Released at Sun May 25 10:50:19 CDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
May 25, 2014 11:00:07 AM SEVERE hudson.remoting.ExportTable unexportByOid Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 23 iota=171 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.Exception: Object was recently deallocated #23 (ref.0) : hudson.CloseProofOutputStream Created at Sun May 25 10:28:30 CDT 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:161) at sun.reflect.GeneratedMethodAccessor141.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.call(Channel.java:738) at hudson.FilePath.act(FilePath.java:909) at hudson.FilePath.act(FilePath.java:893) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:953) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:681) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:518) at hudson.model.Run.execute(Run.java:1706) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Released at Sun May 25 10:32:04 CDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 13 more Caused by: Released at Sun May 25 10:32:04 CDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
We're seeing this error as well after upgrading from 1.551 (Java 6) to 1.565 (Java 7), though I haven't run into any issues so far aside from a lot of log spam:
SEVERE: Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 123 iota=252 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: java.lang.Exception: Object was recently deallocated #123 (ref.0) : hudson.CloseProofOutputStream Created at Tue May 27 11:15:27 MDT 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor467.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:161) at sun.reflect.GeneratedMethodAccessor477.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.call(Channel.java:738) at hudson.FilePath.act(FilePath.java:909) at hudson.FilePath.act(FilePath.java:893) at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:66) at hudson.plugins.git.GitSCM.createClient(GitSCM.java:566) at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:522) at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:457) at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:357) at hudson.scm.SCM.poll(SCM.java:374) at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1449) at hudson.model.AbstractProject._poll(AbstractProject.java:1420) at hudson.model.AbstractProject.poll(AbstractProject.java:1331) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:466) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:495) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Released at Tue May 27 11:25:34 MDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 5 more Caused by: Released at Tue May 27 11:25:34 MDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)
I am also seeing this on 1.565 (Java 7u51). It might have been there earlier without me noticing.
As requested by Kohsuke Kawaguchi attached is the stack-trace I'm seeing:
Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 197 iota=480 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: java.lang.Exception: Object was recently deallocated #197 (ref.0) : hudson.CloseProofOutputStream Created at Wed Jun 04 16:11:05 BST 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:959) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1480) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1416) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1528) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1416) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:346) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.call(Channel.java:738) at hudson.Launcher$RemoteLauncher.launch(Launcher.java:888) at hudson.Launcher$ProcStarter.start(Launcher.java:355) at hudson.Launcher$ProcStarter.join(Launcher.java:362) at hudson.plugins.msbuild.MsBuildBuilder.perform(MsBuildBuilder.java:180) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:745) at hudson.model.Build$BuildExecution.build(Build.java:198) at hudson.model.Build$BuildExecution.doRun(Build.java:159) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:518) at hudson.model.Run.execute(Run.java:1710) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Released at Wed Jun 04 16:17:53 BST 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 5 more Caused by: Released at Wed Jun 04 16:17:53 BST 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679)
This eventually leads to a bunch of slaves no longer appearing to be connected to our master jenkins, restarting the master does not affect this, and we need to restart the slaves individually. Notably this does not affect all our slaves. (But the ones that are affected, are always the same ones!)
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 69 iota=136
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300)
at hudson.remoting.Channel.unexport(Channel.java:600)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38)
at hudson.remoting.Channel$2.handle(Channel.java:475)
at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:529)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.Exception: Object was recently deallocated
#69 (ref.0) : hudson.CloseProofOutputStream
Created at Mon Jun 09 09:51:00 PDT 2014
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86)
at hudson.remoting.ExportTable.export(ExportTable.java:239)
at hudson.remoting.Channel.export(Channel.java:592)
at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82)
at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:161)
at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at hudson.remoting.UserRequest._serialize(UserRequest.java:155)
at hudson.remoting.UserRequest.serialize(UserRequest.java:164)
at hudson.remoting.UserRequest.<init>(UserRequest.java:62)
at hudson.remoting.Channel.call(Channel.java:738)
at hudson.FilePath.act(FilePath.java:909)
at hudson.FilePath.act(FilePath.java:893)
at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:66)
at hudson.plugins.git.GitSCM.createClient(GitSCM.java:566)
at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:522)
at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:457)
at hudson.scm.SCM._compareRemoteRevisionWith(SCM.java:357)
at hudson.scm.SCM.poll(SCM.java:374)
at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1449)
at hudson.model.AbstractProject._poll(AbstractProject.java:1420)
at hudson.model.AbstractProject.poll(AbstractProject.java:1331)
at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:466)
at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:495)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Released at Mon Jun 09 09:55:00 PDT 2014
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303)
at hudson.remoting.Channel.unexport(Channel.java:600)
at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352)
at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270)
... 17 more
Caused by: Released at Mon Jun 09 09:55:00 PDT 2014
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303)
at hudson.remoting.Channel.unexport(Channel.java:600)
at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352)
at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
... 5 more
Running Jenkins 1.566. The error has shown up so far with Invalid object IDs: 5, 6, 7, 9, 11, 78, 83 and 86. The stack trace is the same regardless.
INFO: SCM changes detected in Ads-QA. Triggering #24 Jun 23, 2014 3:09:14 PM hudson.remoting.ExportTable unexportByOid SEVERE: Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 6 iota=18 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:277) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:300) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: java.lang.Exception: Object was recently deallocated #6 (ref.0) : hudson.remoting.RemoteClassLoader$ClassLoaderProxy Created at Mon Jun 23 14:52:03 UTC 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:86) at hudson.remoting.ExportTable.export(ExportTable.java:239) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.RemoteClassLoader.exportId(RemoteClassLoader.java:676) at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:728) at sun.reflect.GeneratedMethodAccessor2158.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:300) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Released at Mon Jun 23 15:09:12 UTC 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) at hudson.remoting.Channel.unexport(Channel.java:600) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:270) ... 5 more Caused by: Released at Mon Jun 23 15:09:12 UTC 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:115) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:303) ... 4 more
We're also running into the same issue on Jenkins ver. 1.567. This error is now consistent on almost all jobs (7/9) for this build server. The error occurs in a seemingly random spot of the test execution and prevents the job from succeeding. The only things that change in the below trace are the timestamps and the iota (it's either 9 or 10). Trace:
org.apache.maven.InternalErrorException: Internal error: java.lang.IllegalStateException: Invalid object ID 6 iota=9 at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168) at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:178) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at hudson.maven.Maven3Builder.call(Maven3Builder.java:134) at hudson.maven.Maven3Builder.call(Maven3Builder.java:69) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.IllegalStateException: Invalid object ID 6 iota=9 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:304) at hudson.remoting.ExportTable.get(ExportTable.java:288) at hudson.remoting.Channel.getExportedObject(Channel.java:596) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:303) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:292) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:251) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:328) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Caused by: java.lang.Exception: Object was recently deallocated #6 (ref.0) : com.sun.proxy.$Proxy50 Created at Thu Jun 26 10:30:15 PDT 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:91) at hudson.remoting.ExportTable.export(ExportTable.java:266) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.Channel.export(Channel.java:583) at hudson.remoting.Channel.export(Channel.java:553) at hudson.maven.MavenBuild$ProxyImpl2.writeReplace(MavenBuild.java:735) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1075) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1132) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at java.util.HashMap.writeObject(HashMap.java:1100) at sun.reflect.GeneratedMethodAccessor434.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at hudson.remoting.UserRequest._serialize(UserRequest.java:155) at hudson.remoting.UserRequest.serialize(UserRequest.java:164) at hudson.remoting.UserRequest.<init>(UserRequest.java:62) at hudson.remoting.Channel.call(Channel.java:738) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:160) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:839) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1710) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:525) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:234) Released at Thu Jun 26 11:58:27 PDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125) at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:212) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:190) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:186) at hudson.remoting.Channel.call(Channel.java:757) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:160) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:839) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1710) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:525) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:234) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:297) ... 15 more Caused by: Released at Thu Jun 26 11:58:27 PDT 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125) at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:212) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:190) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:186) at hudson.remoting.Channel.call(Channel.java:757) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:160) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:839) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1710) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:525) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:234)
Update from my previous comment:
Upgrading the versions of the swarm plugin and the slave-client jar both to 1.16 has fixed the problem for me. It's been a week since I have seen this error.
Thanks everyone for attaching the stack trace, but you are cutting out the crucial part of the stack trace! Specifically, we need to see "unexportByOid2nd 2nd unexport attempt is here" message that immediately follows this.
I've subsequently made this fix in remoting 2.42 that I believe fixes the majority of these problems. Check if you have any slave that's running slave.jar older than 2.42, and upgrade.
If you continue to see this problem despite slave.jar being newer, please attach the whole stack traces — 2 of them to be exact.
Remoting 2.42 was integrated in Jenkins 1.567. VersionColumn Plugin helps determine whether you're on the correct version of remoting/slave.jar. It adds a column to the /computer page with the version installed on the slave (and marks outdated slaves offline).
Code changed in jenkins
User: Kohsuke Kawaguchi
Path:
src/main/java/hudson/remoting/RemoteInvocationHandler.java
src/main/java/hudson/remoting/UnexportCommand.java
http://jenkins-ci.org/commit/remoting/16b9994261cc80f6af8220299e50e827e3b57d11
Log:
JENKINS-22853
Still more diagnostics to keep track of where RemoteInvocationHandler came from.
Jenkins version 1.595:
SEVERE: Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 79 iota=97 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:354) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:377) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$2.handle(Channel.java:483) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: java.lang.Exception: Object was recently deallocated #79 (ref.0) : object=null type=hudson.CloseProofOutputStream interfaces=[java.io.OutputStream] Created at Tue Dec 23 10:29:00 EST 2014 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99) at hudson.remoting.ExportTable.export(ExportTable.java:305) at hudson.remoting.Channel.internalExport(Channel.java:600) at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82) at sun.reflect.GeneratedMethodAccessor181.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:940) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330) at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:161) at sun.reflect.GeneratedMethodAccessor180.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:940) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330) at hudson.remoting.UserRequest._serialize(UserRequest.java:158) at hudson.remoting.UserRequest.serialize(UserRequest.java:167) at hudson.remoting.UserRequest.<init>(UserRequest.java:63) at hudson.remoting.Channel.call(Channel.java:750) at hudson.FilePath.act(FilePath.java:970) at hudson.FilePath.act(FilePath.java:959) at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:66) at hudson.plugins.git.GitSCM.createClient(GitSCM.java:637) at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:590) at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:516) at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:380) at hudson.scm.SCM.poll(SCM.java:397) at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1462) at hudson.model.AbstractProject._poll(AbstractProject.java:1433) at hudson.model.AbstractProject.poll(AbstractProject.java:1344) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:515) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:544) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Released at Tue Dec 23 10:33:01 EST 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: Command ProxyOutputStream.Unexport(79) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330) at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:347) ... 5 more Caused by: Released at Tue Dec 23 10:33:01 EST 2014 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:352) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:111) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: Command ProxyOutputStream.Unexport(79) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330) at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172) Dec 23, 2014 10:39:02 AM hudson.remoting.ExportTable unexportByOid SEVERE: 2nd unexport attempt is here Command hudson.remoting.UnexportCommand@2c7f8ec8 created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) at java.lang.ref.Finalizer.access$100(Finalizer.java:14) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172)
same issue here:
Jenkins 1.580.1
Jan 28, 2015 7:50:13 PM hudson.triggers.SCMTrigger$Runner run
INFO: SCM changes detected in XDK-Ide. Triggering #141
Jan 28, 2015 7:50:13 PM hudson.plugins.git.GitTool$DescriptorImpl getInstallation
WARNING: invalid gitTool selection Default
Jan 28, 2015 7:50:14 PM hudson.remoting.ExportTable unexportByOid
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 1614 iota=104979
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:348)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:371)
at hudson.remoting.Channel.unexport(Channel.java:612)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38)
at hudson.remoting.Channel$2.handle(Channel.java:483)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: java.lang.Exception: Object appears to be deallocated at lease before Wed Jan 28 18:30:13 CET 2015
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:344)
... 5 more
Jan 28, 2015 7:50:14 PM hudson.remoting.ExportTable unexportByOid
SEVERE: 2nd unexport attempt is here
Command hudson.remoting.UnexportCommand@15d7a67 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:197)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
I'm seeing this issue for parameterized builds. If I disable parameters everything works smoothly - no errors. I'm running 1.565.2.
Its strange. The builds are running successfully for a couple of days but after that I'm getting this error in Jenkins console.
ERROR: Error fetching remote repo 'origin'
ERROR: Error fetching remote repo 'origin'
When I look for server logs, I see below;
Jan 30, 2015 7:05:17 PM hudson.triggers.SCMTrigger$Runner run
INFO: SCM changes detected in XDK-Ide. Triggering #163
Jan 30, 2015 7:05:24 PM hudson.remoting.ExportTable unexportByOid
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 102 iota=12324
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:348)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:371)
at hudson.remoting.Channel.unexport(Channel.java:612)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38)
at hudson.remoting.Channel$2.handle(Channel.java:483)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: java.lang.Exception: Object appears to be deallocated at lease before Fri Jan 30 16:15:36 CET 2015
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:344)
... 5 more
Jan 30, 2015 7:05:24 PM hudson.remoting.ExportTable unexportByOid
SEVERE: 2nd unexport attempt is here
Command hudson.remoting.UnexportCommand@1fe8ba9 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221)
at java.lang.System$2.invokeFinalize(Unknown Source)
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
After restarting the slave, there was no errors again. Any help ?
still happening for me even with the latest version of jenkins 1.598 and Java8u31
java.lang.IllegalStateException: Invalid object ID 1078 iota=1079
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:354)
at hudson.remoting.ExportTable.get(ExportTable.java:330)
at hudson.remoting.Channel.getExportedObject(Channel.java:604)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:745)
at ......remote call to Palio3_Ctrl_Builds_Phonak_CH01WW7441(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
at hudson.remoting.Channel.call(Channel.java:752)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:179)
at com.sun.proxy.$Proxy92.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:979)
at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536)
at hudson.model.Run.execute(Run.java:1718)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:89)
at hudson.model.Executor.run(Executor.java:240)
Caused by: java.lang.Exception: Object was recently deallocated
#1078 (ref.0) : object=null type=hudson.Launcher$RemoteLaunchCallable$1 interfaces=[hudson.Launcher$RemoteProcess]
Created at Tue Feb 10 13:21:05 CET 2015
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99)
at hudson.remoting.ExportTable.export(ExportTable.java:305)
at hudson.remoting.Channel.internalExport(Channel.java:600)
at hudson.remoting.Channel.export(Channel.java:591)
at hudson.remoting.Channel.export(Channel.java:561)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:745)
Released at Tue Feb 10 13:21:05 CET 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382)
at hudson.remoting.Channel.unexport(Channel.java:612)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$2.handle(Channel.java:483)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command hudson.remoting.UnexportCommand@7d0fdd32 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240)
at java.lang.System$2.invokeFinalize(System.java:1267)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@436 was created for interface hudson.Launcher$RemoteProcess
at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99)
at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111)
at hudson.remoting.Channel.export(Channel.java:592)
at hudson.remoting.Channel.export(Channel.java:561)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:745)
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:347)
at hudson.remoting.ExportTable.get(ExportTable.java:330)
at hudson.remoting.Channel.getExportedObject(Channel.java:604)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:745)
Caused by: Released at Tue Feb 10 13:21:05 CET 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382)
at hudson.remoting.Channel.unexport(Channel.java:612)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$2.handle(Channel.java:483)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command hudson.remoting.UnexportCommand@4a20272c created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240)
at java.lang.System$2.invokeFinalize(System.java:1267)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@436 was created for interface hudson.Launcher$RemoteProcess
at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99)
at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111)
at hudson.remoting.Channel.export(Channel.java:592)
at hudson.remoting.Channel.export(Channel.java:561)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:745)
Seeing this multiple times per day (on heavy use even per hour). I have a setup with 6 slaves with 6 build processors, all running freestyle shell jobs. I build around 400 jobs per hour and have 2 out of 500 fails because of this problem. Please let me know if you need more information.
Example backtrace (this is 1.599 with slave ver. 2.49):
java.lang.IllegalStateException: Invalid object ID 13906 iota=13907
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:354)
at hudson.remoting.ExportTable.get(ExportTable.java:330)
at hudson.remoting.Channel.getExportedObject(Channel.java:604)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ......remote call to otto(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
at hudson.remoting.Channel.call(Channel.java:752)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:179)
at com.sun.proxy.$Proxy66.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:979)
at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536)
at hudson.model.Run.execute(Run.java:1718)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:89)
at hudson.model.Executor.run(Executor.java:240)
Caused by: java.lang.Exception: Object was recently deallocated
#13906 (ref.0) : object=null type=hudson.Launcher$RemoteLaunchCallable$1 interfaces=[hudson.Launcher$RemoteProcess]
Created at Thu Feb 19 11:24:20 CET 2015
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99)
at hudson.remoting.ExportTable.export(ExportTable.java:305)
at hudson.remoting.Channel.internalExport(Channel.java:600)
at hudson.remoting.Channel.export(Channel.java:591)
at hudson.remoting.Channel.export(Channel.java:561)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Released at Thu Feb 19 11:24:20 CET 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382)
at hudson.remoting.Channel.unexport(Channel.java:612)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$2.handle(Channel.java:483)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command hudson.remoting.UnexportCommand@1e039395 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240)
at java.lang.System$2.invokeFinalize(System.java:1267)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@3652 was created for interface hudson.Launcher$RemoteProcess
at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99)
at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111)
at hudson.remoting.Channel.export(Channel.java:592)
at hudson.remoting.Channel.export(Channel.java:561)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:347)
at hudson.remoting.ExportTable.get(ExportTable.java:330)
at hudson.remoting.Channel.getExportedObject(Channel.java:604)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: Released at Thu Feb 19 11:24:20 CET 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382)
at hudson.remoting.Channel.unexport(Channel.java:612)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$2.handle(Channel.java:483)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command hudson.remoting.UnexportCommand@40e49887 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240)
at java.lang.System$2.invokeFinalize(System.java:1267)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@3652 was created for interface hudson.Launcher$RemoteProcess
at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99)
at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111)
at hudson.remoting.Channel.export(Channel.java:592)
at hudson.remoting.Channel.export(Channel.java:561)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:324)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Build step 'Execute shell' marked build as failure
Another stack trace
java.lang.IllegalStateException: Invalid object ID 711 iota=712 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:354) at hudson.remoting.ExportTable.get(ExportTable.java:330) at hudson.remoting.Channel.getExportedObject(Channel.java:604) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) at ......remote call to vme-sv-exe-c13(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) at hudson.remoting.Channel.call(Channel.java:752) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:179) at com.sun.proxy.$Proxy73.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:979) at hudson.Launcher$ProcStarter.join(Launcher.java:388) at org.jenkinsci.plugins.managedscripts.ScriptBuildStep.perform(ScriptBuildStep.java:177) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533) at hudson.model.Run.execute(Run.java:1759) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:89) at hudson.model.Executor.run(Executor.java:240) Caused by: java.lang.Exception: Object was recently deallocated #711 (ref.0) : object=null type=hudson.Launcher$RemoteLaunchCallable$1 interfaces=[hudson.Launcher$RemoteProcess] Created at Wed Apr 15 07:46:40 MST 2015 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99) at hudson.remoting.ExportTable.export(ExportTable.java:305) at hudson.remoting.Channel.internalExport(Channel.java:600) at hudson.remoting.Channel.export(Channel.java:591) at hudson.remoting.Channel.export(Channel.java:561) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) Released at Wed Apr 15 07:46:40 MST 2015 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$2.handle(Channel.java:483) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: Command hudson.remoting.UnexportCommand@d76dbf created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240) at java.lang.System$2.invokeFinalize(Unknown Source) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@2c7 was created for interface hudson.Launcher$RemoteProcess at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99) at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.Channel.export(Channel.java:561) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:347) at hudson.remoting.ExportTable.get(ExportTable.java:330) at hudson.remoting.Channel.getExportedObject(Channel.java:604) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) Caused by: Released at Wed Apr 15 07:46:40 MST 2015 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:382) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$2.handle(Channel.java:483) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: Command hudson.remoting.UnexportCommand@15b6d76 created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240) at java.lang.System$2.invokeFinalize(Unknown Source) at java.lang.ref.Finalizer.runFinalizer(Unknown Source) at java.lang.ref.Finalizer.access$100(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@2c7 was created for interface hudson.Launcher$RemoteProcess at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99) at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111) at hudson.remoting.Channel.export(Channel.java:592) at hudson.remoting.Channel.export(Channel.java:561) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source)
We are currently at Jenkins ver. 1.509.4 but planning to update to the latest LTS 1.596.2.
We installed the latest LTS on our Dev Box and started to see the above. Should I still go ahead with the upgrade or wait for the fix?
How frequent are the job failures?
Should the severity of the issues be raised to blocker?
We are running one jenkins (1.596.2) with 30 swarm slaves (1.22) which have processed around 8500 builds in the last two weeks. We have seen this issue popping up several times (~20).
The stack traces more or less all look the same and are pretty similar to the one from Alex:
FATAL: Invalid object ID 1693 iota=1694 java.lang.IllegalStateException: Invalid object ID 1693 iota=1694 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:348) at hudson.remoting.ExportTable.get(ExportTable.java:324) at hudson.remoting.Channel.getExportedObject(Channel.java:604) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:311) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:295) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:254) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Thread.java:745) at ......remote call to jkslave1971(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) at hudson.remoting.Channel.call(Channel.java:752) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:179) at com.sun.proxy.$Proxy45.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:979) at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533) at hudson.model.Run.execute(Run.java:1759) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:89) at hudson.model.Executor.run(Executor.java:240) Caused by: java.lang.Exception: Object was recently deallocated #1693 (ref.0) : [hudson.Launcher$RemoteProcess] Created at Sat May 09 09:00:27 CEST 2015 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:93) at hudson.remoting.ExportTable.export(ExportTable.java:299) at hudson.remoting.Channel.internalExport(Channel.java:600) at hudson.remoting.Channel.export(Channel.java:591) at hudson.remoting.Channel.export(Channel.java:561) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1138) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1101) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Thread.java:745) Released at Sat May 09 09:00:27 CEST 2015 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:376) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:483) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: Command hudson.remoting.UnexportCommand@3b538dce created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240) at java.lang.System$2.invokeFinalize(System.java:1270) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98) at java.lang.ref.Finalizer.access$100(Finalizer.java:34) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:341) at hudson.remoting.ExportTable.get(ExportTable.java:324) at hudson.remoting.Channel.getExportedObject(Channel.java:604) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:311) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:295) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:254) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Thread.java:745) Caused by: Released at Sat May 09 09:00:27 CEST 2015 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:125) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:376) at hudson.remoting.Channel.unexport(Channel.java:612) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:38) at hudson.remoting.Channel$2.handle(Channel.java:483) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: Command hudson.remoting.UnexportCommand@4ffe6760 created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240) at java.lang.System$2.invokeFinalize(System.java:1270) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98) at java.lang.ref.Finalizer.access$100(Finalizer.java:34) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
It would be great if we could help getting this solved. Please let us know if you need some additional information or if we could try anything else to help. Thanks a lot in advance.
I opened a [ticket with CloudBees] support over this. This trace so fills our logs it's hard to find anything of value.
CloudBees noticed that some of our jenkins slaves were out of date (this was due to our launch method). I have updated all of our slaves and will re-examine the logs after restart.
*It might be that you are getting this error due to out of date slave.jar files.
*
Here is a snip of ONE entry:
Sep 10, 2015 3:14:16 AM hudson.remoting.ExportTable unexportByOid
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 25 iota=36
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:409)
at hudson.remoting.Channel.unexport(Channel.java:638)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$1.handle(Channel.java:498)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: java.lang.Exception: Object was recently deallocated
#25 (ref.0) : object=null type=hudson.CloseProofOutputStream interfaces=[java.io.OutputStream]
Created at Wed Sep 09 07:28:10 UTC 2015
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99)
at hudson.remoting.ExportTable.export(ExportTable.java:305)
at hudson.remoting.Channel.internalExport(Channel.java:626)
at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82)
at sun.reflect.GeneratedMethodAccessor380.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at hudson.remoting.UserRequest._serialize(UserRequest.java:158)
at hudson.remoting.UserRequest.serialize(UserRequest.java:167)
at hudson.remoting.UserRequest.<init>(UserRequest.java:63)
at hudson.remoting.Channel.call(Channel.java:776)
at hudson.Launcher$RemoteLauncher.launch(Launcher.java:929)
at hudson.Launcher$ProcStarter.start(Launcher.java:382)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
at hudson.model.Run.execute(Run.java:1741)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:381)
Released at Thu Sep 10 03:14:16 UTC 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414)
at hudson.remoting.Channel.unexport(Channel.java:638)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$1.handle(Channel.java:498)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: Command hudson.remoting.UnexportCommand@51000fd4 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:222)
at java.lang.System$2.invokeFinalize(System.java:1213)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379)
... 5 more
Caused by: Released at Thu Sep 10 03:14:16 UTC 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414)
... 4 more
Caused by: Command hudson.remoting.UnexportCommand@51000fd4 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:222)
at java.lang.System$2.invokeFinalize(System.java:1213)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Sep 10, 2015 3:14:16 AM hudson.remoting.ExportTable unexportByOid
SEVERE: 2nd unexport attempt is here
Command hudson.remoting.UnexportCommand@279f02f6 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:222)
at java.lang.System$2.invokeFinalize(System.java:1213)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:98)
at java.lang.ref.Finalizer.access$100(Finalizer.java:34)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:210)
Appears periodically on 1.609.3 with 2.52 remoting on JDK 1.8.0_51.
org.apache.maven.InternalErrorException: Internal error: java.lang.IllegalStateException: Invalid object ID 7 iota=13 at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168) at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:331) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:181) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at hudson.maven.Maven3Builder.call(Maven3Builder.java:136) at hudson.maven.Maven3Builder.call(Maven3Builder.java:71) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalStateException: Invalid object ID 7 iota=13 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386) at hudson.remoting.ExportTable.get(ExportTable.java:330) at hudson.remoting.Channel.getExportedObject(Channel.java:630) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:599) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) at ......remote call to channel(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413) at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) at hudson.remoting.Channel.call(Channel.java:778) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250) at com.sun.proxy.$Proxy5.isArchivingDisabled(Unknown Source) at hudson.maven.MavenBuildProxy$Filter.isArchivingDisabled(MavenBuildProxy.java:247) at hudson.maven.reporters.MavenArtifact.archive(MavenArtifact.java:328) at hudson.maven.reporters.MavenArtifactArchiver.postBuild(MavenArtifactArchiver.java:106) at hudson.maven.Maven3Builder$MavenExecutionListener.sessionEnded(Maven3Builder.java:471) at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire(DefaultExecutionEventCatapult.java:64) at org.apache.maven.lifecycle.internal.DefaultExecutionEventCatapult.fire(DefaultExecutionEventCatapult.java:42) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:170) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) ... 22 more Caused by: java.lang.Exception: Object was recently deallocated #7 (ref.0) : object=null type=com.sun.proxy.$Proxy56 interfaces=[hudson.maven.MavenBuildProxy2] Created at Tue Sep 15 20:22:12 EDT 2015 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99) at hudson.remoting.ExportTable.export(ExportTable.java:305) at hudson.remoting.Channel.internalExport(Channel.java:626) at hudson.remoting.Channel.export(Channel.java:617) at hudson.remoting.Channel.export(Channel.java:587) at hudson.maven.MavenBuild$ProxyImpl2.writeReplace(MavenBuild.java:751) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1075) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1134) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at java.util.HashMap.writeObject(HashMap.java:1129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at hudson.remoting.UserRequest._serialize(UserRequest.java:158) at hudson.remoting.UserRequest.serialize(UserRequest.java:167) at hudson.remoting.UserRequest.<init>(UserRequest.java:63) at hudson.remoting.Channel.call(Channel.java:776) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:161) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:853) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) at hudson.model.Run.execute(Run.java:1738) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:381) Released at Wed Sep 16 02:22:06 EDT 2015 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:247) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:193) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:189) at hudson.remoting.Channel.call(Channel.java:795) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:161) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:853) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) at hudson.model.Run.execute(Run.java:1738) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:381) at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379) at hudson.remoting.ExportTable.get(ExportTable.java:330) at hudson.remoting.Channel.getExportedObject(Channel.java:630) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:599) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ... 1 more Caused by: Released at Wed Sep 16 02:22:06 EDT 2015 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:247) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:193) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:189) at hudson.remoting.Channel.call(Channel.java:795) at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:161) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:853) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) at hudson.model.Run.execute(Run.java:1738) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:381)
Seeing this on 1.609.3 as well - master is on 1.7.0_55, slave is on 1.7.0_25.
I am also seeing these errors on rare occasions with Jenkins 1.620 with JRE 1.8.0_60-b27 and 1.7.0_71-b14 on slaves
02:36:13 ERROR: Build step failed with exception 02:36:13 java.lang.IllegalStateException: Invalid object ID 7862 iota=7863 02:36:13 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386) 02:36:13 at hudson.remoting.ExportTable.get(ExportTable.java:330) 02:36:13 at hudson.remoting.Channel.getExportedObject(Channel.java:605) 02:36:13 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317) 02:36:13 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301) 02:36:13 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 02:36:13 at hudson.remoting.Request$2.run(Request.java:325) 02:36:13 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 02:36:13 at java.util.concurrent.FutureTask.run(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02:36:13 at hudson.remoting.Engine$1$1.run(Engine.java:69) 02:36:13 at java.lang.Thread.run(Unknown Source) 02:36:13 at ......remote call to beyqbbuild101(Native Method) 02:36:13 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1361) 02:36:13 at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) 02:36:13 at hudson.remoting.Channel.call(Channel.java:753) 02:36:13 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:179) 02:36:13 at com.sun.proxy.$Proxy70.join(Unknown Source) 02:36:13 at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:992) 02:36:13 at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137) 02:36:13 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) 02:36:13 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) 02:36:13 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 02:36:13 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) 02:36:13 at hudson.model.Build$BuildExecution.build(Build.java:205) 02:36:13 at hudson.model.Build$BuildExecution.doRun(Build.java:162) 02:36:13 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537) 02:36:13 at hudson.model.Run.execute(Run.java:1741) 02:36:13 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 02:36:13 at hudson.model.ResourceController.execute(ResourceController.java:98) 02:36:13 at hudson.model.Executor.run(Executor.java:381) 02:36:13 Caused by: java.lang.Exception: Object was recently deallocated 02:36:13 #7862 (ref.0) : object=null type=hudson.Launcher$RemoteLaunchCallable$1 interfaces=[hudson.Launcher$RemoteProcess] 02:36:13 Created at Mon Oct 19 02:36:11 EDT 2015 02:36:13 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99) 02:36:13 at hudson.remoting.ExportTable.export(ExportTable.java:305) 02:36:13 at hudson.remoting.Channel.internalExport(Channel.java:601) 02:36:13 at hudson.remoting.Channel.export(Channel.java:592) 02:36:13 at hudson.remoting.Channel.export(Channel.java:562) 02:36:13 at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1151) 02:36:13 at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1114) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 02:36:13 at hudson.remoting.Request$2.run(Request.java:325) 02:36:13 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 02:36:13 at java.util.concurrent.FutureTask.run(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02:36:13 at hudson.remoting.Engine$1$1.run(Engine.java:69) 02:36:13 at java.lang.Thread.run(Unknown Source) 02:36:13 Released at Mon Oct 19 02:36:12 EDT 2015 02:36:13 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) 02:36:13 at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414) 02:36:13 at hudson.remoting.Channel.unexport(Channel.java:613) 02:36:13 at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) 02:36:13 at hudson.remoting.Channel$2.handle(Channel.java:484) 02:36:13 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) 02:36:13 Caused by: Command hudson.remoting.UnexportCommand@757afd62 created at 02:36:13 at hudson.remoting.Command.<init>(Command.java:67) 02:36:13 at hudson.remoting.Command.<init>(Command.java:50) 02:36:13 at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) 02:36:13 at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240) 02:36:13 at java.lang.System$2.invokeFinalize(Unknown Source) 02:36:13 at java.lang.ref.Finalizer.runFinalizer(Unknown Source) 02:36:13 at java.lang.ref.Finalizer.access$100(Unknown Source) 02:36:13 at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) 02:36:13 Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@1eb6 was created for interface hudson.Launcher$RemoteProcess 02:36:13 at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99) 02:36:13 at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111) 02:36:13 at hudson.remoting.Channel.export(Channel.java:593) 02:36:13 at hudson.remoting.Channel.export(Channel.java:562) 02:36:13 at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1151) 02:36:13 at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1114) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 02:36:13 at hudson.remoting.Request$2.run(Request.java:325) 02:36:13 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 02:36:13 at java.util.concurrent.FutureTask.run(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02:36:13 at hudson.remoting.Engine$1$1.run(Engine.java:69) 02:36:13 at java.lang.Thread.run(Unknown Source) 02:36:13 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379) 02:36:13 at hudson.remoting.ExportTable.get(ExportTable.java:330) 02:36:13 at hudson.remoting.Channel.getExportedObject(Channel.java:605) 02:36:13 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:317) 02:36:13 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:301) 02:36:13 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:260) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 02:36:13 at hudson.remoting.Request$2.run(Request.java:325) 02:36:13 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 02:36:13 at java.util.concurrent.FutureTask.run(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02:36:13 at hudson.remoting.Engine$1$1.run(Engine.java:69) 02:36:13 at java.lang.Thread.run(Unknown Source) 02:36:13 Caused by: Released at Mon Oct 19 02:36:12 EDT 2015 02:36:13 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) 02:36:13 at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414) 02:36:13 at hudson.remoting.Channel.unexport(Channel.java:613) 02:36:13 at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) 02:36:13 at hudson.remoting.Channel$2.handle(Channel.java:484) 02:36:13 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) 02:36:13 Caused by: Command hudson.remoting.UnexportCommand@3ca0f491 created at 02:36:13 at hudson.remoting.Command.<init>(Command.java:67) 02:36:13 at hudson.remoting.Command.<init>(Command.java:50) 02:36:13 at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) 02:36:13 at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:240) 02:36:13 at java.lang.System$2.invokeFinalize(Unknown Source) 02:36:13 at java.lang.ref.Finalizer.runFinalizer(Unknown Source) 02:36:13 at java.lang.ref.Finalizer.access$100(Unknown Source) 02:36:13 at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) 02:36:13 Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@1eb6 was created for interface hudson.Launcher$RemoteProcess 02:36:13 at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:99) 02:36:13 at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:111) 02:36:13 at hudson.remoting.Channel.export(Channel.java:593) 02:36:13 at hudson.remoting.Channel.export(Channel.java:562) 02:36:13 at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1151) 02:36:13 at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1114) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:121) 02:36:13 at hudson.remoting.UserRequest.perform(UserRequest.java:49) 02:36:13 at hudson.remoting.Request$2.run(Request.java:325) 02:36:13 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) 02:36:13 at java.util.concurrent.FutureTask.run(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 02:36:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 02:36:13 at hudson.remoting.Engine$1$1.run(Engine.java:69) 02:36:13 at java.lang.Thread.run(Unknown Source)
I got this exception after update of Jenkins from 1.620 to 1.634 and Java on master from jdk1.7.0_45 to jdk1.8.0_66. Slaves still use Java SE RE 1.7.0_03-b04.
Slave.jar version: 2.52
I also see this error message on a low percentage of builds, running Jenkins 1.580.2, since I upgraded from Java 7 to 8 a few days ago.
Starting this morning our build system is basically down. All 300 jobs get stuck @ EnvInject and in the master log i see this error over and over again. No update of plugins or system was done. Does anyone have a workaround?
slave version: 2.52
master: 2.52
Jenkins Master version : 1.609.3
JDK: 1.7.0_55
My log:
INFO: SCM changes detected in xxxxxx. Triggering #232
Dec 08, 2015 11:27:01 AM hudson.remoting.ExportTable unexportByOid
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 21 iota=299
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:409)
at hudson.remoting.Channel.unexport(Channel.java:638)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$1.handle(Channel.java:498)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: java.lang.Exception: Object was recently deallocated
#21 (ref.0) : object=null type=hudson.scm.CredentialsSVNAuthenticationProviderImpl$RemotableSVNAuthenticationBuilderProvider interfaces=[hudson.scm.CredentialsSVNAuthenticationProviderImpl$SVNAuthenticationBuilderProvider]
Created at Tue Dec 08 11:22:02 IST 2015
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99)
at hudson.remoting.ExportTable.export(ExportTable.java:305)
at hudson.remoting.Channel.internalExport(Channel.java:626)
at hudson.remoting.Channel.export(Channel.java:617)
at hudson.remoting.Channel.export(Channel.java:587)
at hudson.scm.CredentialsSVNAuthenticationProviderImpl$RemotableSVNAuthenticationBuilderProvider.writeReplace(CredentialsSVNAuthenticationProviderImpl.java:295)
at sun.reflect.GeneratedMethodAccessor300.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1075)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1134)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at java.util.HashMap.writeObject(HashMap.java:1133)
at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at hudson.remoting.UserRequest._serialize(UserRequest.java:158)
at hudson.remoting.UserRequest.serialize(UserRequest.java:167)
at hudson.remoting.UserRequest.<init>(UserRequest.java:63)
at hudson.remoting.Channel.call(Channel.java:776)
at hudson.scm.SubversionSCM.compareRemoteRevisionWith(SubversionSCM.java:1416)
at hudson.scm.SCM.poll(SCM.java:397)
at hudson.model.AbstractProject._poll(AbstractProject.java:1450)
at hudson.model.AbstractProject.poll(AbstractProject.java:1353)
at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510)
at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Released at Tue Dec 08 11:22:14 IST 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:247)
at hudson.remoting.UserRequest.releaseExports(UserRequest.java:193)
at hudson.remoting.UserRequest.releaseExports(UserRequest.java:189)
at hudson.remoting.Channel.call(Channel.java:795)
at hudson.scm.SubversionSCM.compareRemoteRevisionWith(SubversionSCM.java:1416)
at hudson.scm.SCM.poll(SCM.java:397)
at hudson.model.AbstractProject._poll(AbstractProject.java:1450)
at hudson.model.AbstractProject.poll(AbstractProject.java:1353)
at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510)
at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379)
... 5 more
Caused by: Released at Tue Dec 08 11:22:14 IST 2015
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:247)
at hudson.remoting.UserRequest.releaseExports(UserRequest.java:193)
at hudson.remoting.UserRequest.releaseExports(UserRequest.java:189)
at hudson.remoting.Channel.call(Channel.java:795)
at hudson.scm.SubversionSCM.compareRemoteRevisionWith(SubversionSCM.java:1416)
at hudson.scm.SCM.poll(SCM.java:397)
at hudson.model.AbstractProject._poll(AbstractProject.java:1450)
at hudson.model.AbstractProject.poll(AbstractProject.java:1353)
at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:119)
at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510)
at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Dec 08, 2015 11:27:01 AM hudson.remoting.ExportTable unexportByOid
SEVERE: 2nd unexport attempt is here
Command hudson.remoting.UnexportCommand@282c48e7 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:178)
I hit this error but it turned out now that it was caused for me by an infinite loop in a unit test. Might be worth for others too to look whether they see any timeouts in the build log before hitting the stack trace reported in this issue.
Timeout:
Build timed out (after 235 minutes). Marking the build as failed. Results : Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 [ERROR] There was a timeout or other error in the fork
Stack trace later
[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3:54:51.866s [INFO] Finished at: Tue Dec 29 23:01:30 CET 2015 [INFO] Final Memory: 136M/247M [INFO] ------------------------------------------------------------------------ [ERROR] Internal error: java.lang.IllegalStateException: Invalid object ID 41 iota=300: Object was recently deallocated [ERROR] #41 (ref.0) : object=null type=$Proxy53 interfaces=[hudson.maven.MavenBuildProxy2] [ERROR] Created at Tue Dec 29 19:06:33 CET 2015 [ERROR] at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99) [ERROR] at hudson.remoting.ExportTable.export(ExportTable.java:305) [ERROR] at hudson.remoting.Channel.internalExport(Channel.java:626) [ERROR] at hudson.remoting.Channel.export(Channel.java:617) [ERROR] at hudson.remoting.Channel.export(Channel.java:587) [ERROR] at hudson.maven.MavenBuild$ProxyImpl2.writeReplace(MavenBuild.java:751) [ERROR] at sun.reflect.GeneratedMethodAccessor2332.invoke(Unknown Source) [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [ERROR] at java.lang.reflect.Method.invoke(Method.java:597) [ERROR] at java.io.ObjectStreamClass.invokeWriteReplace(ObjectStreamClass.java:1032) [ERROR] at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1115) [ERROR] at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518) [ERROR] at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483) [ERROR] at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) [ERROR] at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) [ERROR] at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330) [ERROR] at java.util.HashMap.writeObject(HashMap.java:1001) [ERROR] at sun.reflect.GeneratedMethodAccessor2336.invoke(Unknown Source) [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [ERROR] at java.lang.reflect.Method.invoke(Method.java:597) [ERROR] at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945) [ERROR] at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469) [ERROR] at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) [ERROR] at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) [ERROR] at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518) [ERROR] at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483) [ERROR] at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400) [ERROR] at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158) [ERROR] at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330) [ERROR] at hudson.remoting.UserRequest._serialize(UserRequest.java:158) [ERROR] at hudson.remoting.UserRequest.serialize(UserRequest.java:167) [ERROR] at hudson.remoting.UserRequest.<init>(UserRequest.java:63) [ERROR] at hudson.remoting.Channel.call(Channel.java:776) [ERROR] at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:161) [ERROR] at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:853) [ERROR] at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) [ERROR] at hudson.model.Run.execute(Run.java:1738) [ERROR] at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531) [ERROR] at hudson.model.ResourceController.execute(ResourceController.java:98) [ERROR] at hudson.model.Executor.run(Executor.java:381) [ERROR] Released at Tue Dec 29 23:01:30 CET 2015 [ERROR] at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) [ERROR] at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:247) [ERROR] at hudson.remoting.UserRequest.releaseExports(UserRequest.java:193) [ERROR] at hudson.remoting.UserRequest.releaseExports(UserRequest.java:189) [ERROR] at hudson.remoting.Channel.call(Channel.java:795) [ERROR] at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:161) [ERROR] at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:853) [ERROR] at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536) [ERROR] at hudson.model.Run.execute(Run.java:1738) [ERROR] at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531) [ERROR] at hudson.model.ResourceController.execute(ResourceController.java:98) [ERROR] at hudson.model.Executor.run(Executor.java:381): ReleasedAt [ERROR] -> [Help 1]
Hi all,
do you need more information to track the bug? This is hitting us very often (every five minutes or so) and spamming our logfiles. Let me know if you need further input. Our Jenkins version is 1.625.2.
Mar 29, 2016 11:22:14 AM hudson.remoting.ExportTable unexportByOid
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 471 iota=948
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:409)
at hudson.remoting.Channel.unexport(Channel.java:641)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$1.handle(Channel.java:501)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: java.lang.Exception: Object was recently deallocated
#471 (ref.0) : object=null type=hudson.CloseProofOutputStream interfaces=[java.io.OutputStream]
Created at Mon Mar 28 15:31:11 CEST 2016
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99)
at hudson.remoting.ExportTable.export(ExportTable.java:305)
at hudson.remoting.Channel.internalExport(Channel.java:629)
at hudson.remoting.RemoteOutputStream.writeObject(RemoteOutputStream.java:82)
at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at hudson.util.StreamTaskListener.writeObject(StreamTaskListener.java:163)
at sun.reflect.GeneratedMethodAccessor160.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at hudson.remoting.UserRequest._serialize(UserRequest.java:157)
at hudson.remoting.UserRequest.serialize(UserRequest.java:166)
at hudson.remoting.UserRequest.<init>(UserRequest.java:62)
at hudson.remoting.Channel.call(Channel.java:779)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250)
at com.sun.proxy.$Proxy40.withRepository(Unknown Source)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl.withRepository(RemoteGitImpl.java:238)
at hudson.plugins.git.util.GitUtils.filterTipBranches(GitUtils.java:127)
at hudson.plugins.git.util.DefaultBuildChooser.getAdvancedCandidateRevisions(DefaultBuildChooser.java:249)
at hudson.plugins.git.util.DefaultBuildChooser.getCandidateRevisions(DefaultBuildChooser.java:53)
at hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(GitSCM.java:580)
at hudson.plugins.git.GitSCM.compareRemoteRevisionWith(GitSCM.java:494)
at hudson.scm.SCM.poll(SCM.java:398)
at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1474)
at hudson.model.AbstractProject._poll(AbstractProject.java:1444)
at hudson.model.AbstractProject.poll(AbstractProject.java:1355)
at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:510)
at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:539)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Released at Mon Mar 28 15:37:15 CEST 2016
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414)
at hudson.remoting.Channel.unexport(Channel.java:641)
at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:334)
at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: Command ProxyOutputStream.Unexport(471) created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330)
at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379)
... 5 more
Caused by: Released at Mon Mar 28 15:37:15 CEST 2016
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414)
at hudson.remoting.Channel.unexport(Channel.java:641)
at hudson.remoting.ProxyOutputStream$Unexport$1.run(ProxyOutputStream.java:334)
at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: Command ProxyOutputStream.Unexport(471) created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.ProxyOutputStream$Unexport.<init>(ProxyOutputStream.java:330)
at hudson.remoting.ProxyOutputStream.finalize(ProxyOutputStream.java:187)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)Mar 29, 2016 11:22:14 AM hudson.remoting.ExportTable unexportByOid
SEVERE: 2nd unexport attempt is here
Command hudson.remoting.UnexportCommand@317fa8b9 created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33)
at hudson.remoting.RemoteInvocationHandler.finalize(RemoteInvocationHandler.java:221)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Unknown Source)
at java.lang.ref.Finalizer.access$100(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
Sorry, forgot to mention our Java version:
java version "1.7.0_95"
OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-1~deb7u1)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
The standard on Debian Wheezy.
Hitting this as well on Centos 7.2.1511 with
java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
Below is the full stack trace, in case it's relevant (some kind of write-race condition) a separate node from the same machine had a long-running job at the time that was spitting lots of output into its console.
ERROR: Build step failed with exception java.lang.IllegalStateException: Invalid object ID 34826 iota=34827 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386) at hudson.remoting.ExportTable.get(ExportTable.java:330) at hudson.remoting.Channel.getExportedObject(Channel.java:633) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:599) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542) at hudson.remoting.UserRequest.perform(UserRequest.java:120) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at ......remote call to idata-foundry-01.jira(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) at hudson.remoting.UserResponse.retrieve(UserRequest.java:220) at hudson.remoting.Channel.call(Channel.java:781) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:250) at com.sun.proxy.$Proxy51.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:991) at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:135) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:95) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:64) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782) at hudson.model.Build$BuildExecution.build(Build.java:205) at hudson.model.Build$BuildExecution.doRun(Build.java:162) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1738) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) Caused by: java.lang.Exception: Object was recently deallocated #34826 (ref.0) : object=null type=hudson.Launcher$RemoteLaunchCallable$1 interfaces=[hudson.Launcher$RemoteProcess] Created at Thu Mar 31 06:06:00 PDT 2016 at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99) at hudson.remoting.ExportTable.export(ExportTable.java:305) at hudson.remoting.Channel.internalExport(Channel.java:629) at hudson.remoting.Channel.export(Channel.java:620) at hudson.remoting.Channel.export(Channel.java:590) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1150) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1113) at hudson.remoting.UserRequest.perform(UserRequest.java:120) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Released at Thu Mar 31 06:06:00 PDT 2016 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414) at hudson.remoting.Channel.unexport(Channel.java:641) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$1.handle(Channel.java:501) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: Command hudson.remoting.UnexportCommand@13f94d85 created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler$PhantomReferenceImpl.cleanup(RemoteInvocationHandler.java:360) at hudson.remoting.RemoteInvocationHandler$PhantomReferenceImpl.access$700(RemoteInvocationHandler.java:319) at hudson.remoting.RemoteInvocationHandler$Unexporter.run(RemoteInvocationHandler.java:420) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:110) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@880a was created for interface hudson.Launcher$RemoteProcess at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:125) at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:137) at hudson.remoting.Channel.export(Channel.java:621) at hudson.remoting.Channel.export(Channel.java:590) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1150) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1113) at hudson.remoting.UserRequest.perform(UserRequest.java:120) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379) at hudson.remoting.ExportTable.get(ExportTable.java:330) at hudson.remoting.Channel.getExportedObject(Channel.java:633) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:599) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:583) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:542) at hudson.remoting.UserRequest.perform(UserRequest.java:120) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: Released at Thu Mar 31 06:06:00 PDT 2016 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414) at hudson.remoting.Channel.unexport(Channel.java:641) at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43) at hudson.remoting.Channel$1.handle(Channel.java:501) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: Command hudson.remoting.UnexportCommand@56cf798c created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.UnexportCommand.<init>(UnexportCommand.java:33) at hudson.remoting.RemoteInvocationHandler$PhantomReferenceImpl.cleanup(RemoteInvocationHandler.java:360) at hudson.remoting.RemoteInvocationHandler$PhantomReferenceImpl.access$700(RemoteInvocationHandler.java:319) at hudson.remoting.RemoteInvocationHandler$Unexporter.run(RemoteInvocationHandler.java:420) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:110) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.Exception: Proxy hudson.remoting.RemoteInvocationHandler@880a was created for interface hudson.Launcher$RemoteProcess at hudson.remoting.RemoteInvocationHandler.<init>(RemoteInvocationHandler.java:125) at hudson.remoting.RemoteInvocationHandler.wrap(RemoteInvocationHandler.java:137) at hudson.remoting.Channel.export(Channel.java:621) at hudson.remoting.Channel.export(Channel.java:590) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1150) at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1113) at hudson.remoting.UserRequest.perform(UserRequest.java:120) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more
Our Jenkins Master and Slave versions were running different versions of the JDK/JRE. (Specifically some slaves were 1.6!). After updating all slaves to 1.7 we are not seeing the Unexport problem any more we have even moved the master to OpenJDK 1.8 and it seems to be getting along just fine with Oracle JDK/JRE 1.7 on most slaves.
We are now getting hammered by a different mega stack trace: MalformedURLException: no protocol:
But the unexport exceptions are gone.
Is matching java versions between master/slave a prerequisite or expectation for full support?
I'm running into the same issue on Jenkins 2.9 and OpenJDK 1.8.0_91, not master/slave.
Jenkins's log
When I tried to do something with Jenkins CLI, the logs of this instance says:
Jun 20, 2016 1:24:05 PM hudson.init.impl.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler uncaughtException SEVERE: A thread (Thread-8/63) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code. org.acegisecurity.userdetails.UsernameNotFoundException: admin at hudson.security.SecurityRealm$None$2.loadUserByUsername(SecurityRealm.java:549) at jenkins.security.ImpersonatingUserDetailsService.loadUserByUsername(ImpersonatingUserDetailsService.java:32) at hudson.model.User.impersonate(User.java:311) at org.jenkinsci.main.modules.cli.auth.ssh.SshCliAuthenticator.authenticate(SshCliAuthenticator.java:44) at hudson.cli.CliManagerImpl$2.run(CliManagerImpl.java:109) Jun 20, 2016 1:24:05 PM hudson.remoting.ExportTable unexportByOid SEVERE: Trying to unexport an object that's already unexported java.lang.IllegalStateException: Invalid object ID 4 iota=5 at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386) at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:409) at hudson.remoting.Channel.unexport(Channel.java:641) at hudson.remoting.ProxyOutputStream$EOF$1.run(ProxyOutputStream.java:366) at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at hudson.cli.CliManagerImpl$1.call(CliManagerImpl.java:63) at hudson.remoting.CallableDecoratorAdapter.call(CallableDecoratorAdapter.java:18) at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Jun 20, 2016 1:24:05 PM hudson.remoting.ExportTable unexportByOid SEVERE: 2nd unexport attempt is here Command ProxyOutputStream.EOF(4) created at at hudson.remoting.Command.<init>(Command.java:67) at hudson.remoting.Command.<init>(Command.java:50) at hudson.remoting.ProxyOutputStream$EOF.<init>(ProxyOutputStream.java:355) at hudson.remoting.ProxyOutputStream.doClose(ProxyOutputStream.java:171) at hudson.remoting.ProxyOutputStream.error(ProxyOutputStream.java:166) at hudson.remoting.ProxyOutputStream.close(ProxyOutputStream.java:157) at hudson.cli.Connection.close(Connection.java:265) at hudson.cli.CLI.authenticate(CLI.java:575) at hudson.cli.CLI._main(CLI.java:478) at hudson.cli.CLI.main(
Situation
I provisioned Jenkins into CentOS 7 with ansible, based on this codes. The provisioned instance loads some groovy scripts in booting it.
import jenkins.* import jenkins.model.* import hudson.* import hudson.model.* def userId = 'admin' def jenkinUserKey = "SSH public key appears here" admin = User.get(userId) def keysProperty = admin.getProperty(org.jenkinsci.main.modules.cli.auth.ssh.UserPropertyImpl) if (keysProperty == null) { keysProperty = new org.jenkinsci.main.modules.cli.auth.ssh.UserPropertyImpl("") } println(keysProperty.authorizedKeys) if (!keysProperty.authorizedKeys.contains(jenkinUserKey)) { keysProperty.authorizedKeys = jenkinUserKey + "\n" + keysProperty.authorizedKeys admin.addProperty(keysProperty) admin.save() println("Added SSH key for admin") } else { println("SSH key was already added for admin") }
And one of the init scripts sets the configuration so that slave Agent's port is choosen randomly.
import jenkins.* import jenkins.model.* import hudson.* import hudson.model.* Jenkins.instance.slaveAgentPort = 0
When I provisioned the instance of Jenkins 2.8, I had never seen this issue. so I will try out Jenkins 2.8 again.
Had the same problem on a Windows slave machine. Updated the slave.jar to 2.43 and the issue was resolved.
From what I see, the issue happens because Jenkins tries to unexport ProxyInputStream objects from two sides.
My repro steps:
1) Create a test slave with 20 executors
2) Run the Pipeline script below
3) Interrupt build once it starts performing short stashes
// The map we'll store the parallel steps in before executing hem. def stepsForParallel = [:] stage "Prepare stashed file" node("test") { sh "cp /Users/nenashev/Documents/jenkins/demo/jenkins-2.7.1/jenkins.war ." for (int i=0; i<10; i++) { sh "cp jenkins.war jenkins_${i}.war" } writeFile file: 'foo.txt', text: 'Hello' stash includes: 'foo.txt', name: 'foo_stashed' stash includes: '*', name: 'jenkins_stashed' } stage "Parallel unstash" for (int i = 0; i < 19; i++) { def step = { node("test") { ws("ws_$i") { unstash includes: '*', name: 'jenkins_stashed' } } } stepsForParallel["unstash_$i"] = step } stepsForParallel["fast_unstash"] = { node("test") { ws("ws_fast") { for (int i=0; i<2000; i++) { unstash includes: '*', name: 'foo_stashed' } } } } parallel stepsForParallel
Side 1. Remote call releases exports at the end of the call. E.g. if it gets interrupted
Released at Thu Aug 04 13:12:16 MSK 2016 at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131) at hudson.remoting.ExportTable$ExportList.release(ExportTable.java:247) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:224) at hudson.remoting.UserRequest.releaseExports(UserRequest.java:220) at hudson.remoting.Channel.call(Channel.java:798) at hudson.FilePath.act(FilePath.java:1007) at hudson.FilePath.act(FilePath.java:996) at hudson.FilePath.untarFrom(FilePath.java:728) at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:114)
Side 2. When ProxyInputStream gets closed, it sends asynchronous EOF command to the remote side. This EOF also tries to release the allocated object (FileInputStream in my case). By the time the request gets to the remote executor, UserRequest.releaseExports() unexports the object.
ProxyInputStream is nested into RemoteInputStream, which is actually being used for many cases like stdin/stdout propagation in remote calls. So technically any remote call operation may overlap with UserRequest termination handler and cause an overlap in such case.
So the solution would be to either make ProxyInputStream EOF command synchronous or to make it tolerant against race conditions.
Reproducing the issue in tests
Code changed in jenkins
User: Bertrand Roussel
Path:
client/pom.xml
http://jenkins-ci.org/commit/swarm-plugin/8ad209aaafad5ec6256aa547702870da083ff6cd
Log:
Upgrade remoting API to 2.62
Version 2.62 contains a fix for the following bugs:
- https://issues.jenkins-ci.org/browse/JENKINS-22853
- https://issues.jenkins-ci.org/browse/JENKINS-23271 (possibly)
This occurs mostly with Java 8.
The remoting version matches with Jenkins 2.18/2.19/2.19.1/2.19.2.
HI There,
We are still seeing this on Jenkins ver. 2.7.1. Is there additional things we can do to troubleshoot. We have also been crashing with permgen errors since upgrade, not sure if related. Please advise if i should reopen this issue. Thanks!
SEVERE: Trying to unexport an object that's already unexported
java.lang.IllegalStateException: Invalid object ID 412 iota=773
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:386)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:409)
at hudson.remoting.Channel.unexport(Channel.java:641)
at hudson.remoting.UnexportCommand.execute(UnexportCommand.java:43)
at hudson.remoting.Channel$1.handle(Channel.java:501)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)
Caused by: java.lang.Exception: Object was recently deallocated
#412 (ref.0) : object=null type=hudson.remoting.FastPipedOutputStream interfaces=[java.lang.Object]
Created at Thu Feb 02 09:24:16 EST 2017
at hudson.remoting.ExportTable$Entry.<init>(ExportTable.java:99)
at hudson.remoting.ExportTable.export(ExportTable.java:305)
at hudson.remoting.Channel.internalExport(Channel.java:629)
at hudson.remoting.Pipe.writeObject(Pipe.java:158)
at sun.reflect.GeneratedMethodAccessor1141.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at hudson.remoting.UserRequest._serialize(UserRequest.java:189)
at hudson.remoting.UserRequest.serialize(UserRequest.java:198)
at hudson.remoting.UserRequest.<init>(UserRequest.java:64)
at hudson.remoting.Channel.callAsync(Channel.java:807)
at hudson.FilePath.actAsync(FilePath.java:1087)
at hudson.FilePath.read(FilePath.java:1782)
at hudson.plugins.groovy.FileScriptSource.getScriptStream(FileScriptSource.java:52)
at hudson.plugins.groovy.SystemGroovy.perform(SystemGroovy.java:93)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1741)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Released at Thu Feb 02 09:24:16 EST 2017
at hudson.remoting.ExportTable$Entry.release(ExportTable.java:131)
at hudson.remoting.ExportTable.unexportByOid(ExportTable.java:414)
at hudson.remoting.Channel.unexport(Channel.java:641)
at hudson.remoting.ProxyOutputStream$EOF$1.run(ProxyOutputStream.java:366)
at hudson.remoting.PipeWriter$1.run(PipeWriter.java:158)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: Command ProxyOutputStream.EOF(412) created at
at hudson.remoting.Command.<init>(Command.java:67)
at hudson.remoting.Command.<init>(Command.java:50)
at hudson.remoting.ProxyOutputStream$EOF.<init>(ProxyOutputStream.java:358)
at hudson.remoting.ProxyOutputStream.doClose(ProxyOutputStream.java:177)
at hudson.remoting.ProxyOutputStream.close(ProxyOutputStream.java:173)
at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:303)
at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:274)
at hudson.FilePath$33.invoke(FilePath.java:1795)
at hudson.FilePath$33.invoke(FilePath.java:1782)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2772)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:328)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
... 1 more
at hudson.remoting.ExportTable.diagnoseInvalidId(ExportTable.java:379)
Code changed in jenkins
User: Kohsuke Kawaguchi
Path:
src/main/java/hudson/remoting/Channel.java
src/main/java/hudson/remoting/ExportTable.java
src/main/java/hudson/remoting/Pipe.java
src/main/java/hudson/remoting/ProxyInputStream.java
src/main/java/hudson/remoting/ProxyOutputStream.java
src/main/java/hudson/remoting/ProxyWriter.java
src/main/java/hudson/remoting/UnexportCommand.java
src/main/java/hudson/remoting/UserRequest.java
src/test/java/hudson/remoting/ExportTableTest.java
http://jenkins-ci.org/commit/remoting/1eeccd431351922ccdb35de1a734f066a13c9d8a
Log:
JENKINS-22853Digging deeper into the excessive unexport problem by adding more
diagnostics.
We want to track where the unexport command is created on the other side
of the channel.