-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
jenkins contoller running on rhel7 at Jenkins 2.332.1 lts version, upgrded to 2.346.1
openstack cloud rhel7 & rhel9 and local linux rhel7 & rhel8 agents both disconnecting
jenkins controller upgrade on Monday a week ago and agent reliability is very bad
sometimes they stop immediately, sometimes they build for 3+ hours
I've reviewd the remoting info for ideas, and will include the agent log
this morning I doubled the controller memory Jenkins
I updated my local and openstack agents to run remoting.jar under the same jdk-11.0.14.1+1 as the controller
the build fails with 16:21:36 FATAL: command execution failed 16:24:49 java.io.EOFException 16:24:49 at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2905) 16:24:49 at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3400) 16:24:49 at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:936) 16:24:49 at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:379) 16:24:49 at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) 16:24:49 at hudson.remoting.Command.readFrom(Command.java:142) 16:24:49 at hudson.remoting.Command.readFrom(Command.java:128) 16:24:49 at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) 16:24:49 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61) 16:24:49 Caused: java.io.IOException: Unexpected termination of the channel 16:24:49 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75) 16:24:49 Caused: java.io.IOException: Backing channel 'mhub-linux9' is disconnected. 16:24:49 at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215) 16:24:49 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285) 16:24:49 at com.sun.proxy.$Proxy105.isAlive(Unknown Source) 16:24:49 at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215) 16:24:49 at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207) 16:24:49 at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:195) 16:24:49 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:145) 16:24:49 at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92) 16:24:49 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 16:24:49 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814) 16:24:49 at hudson.model.Build$BuildExecution.build(Build.java:199) 16:24:49 at hudson.model.Build$BuildExecution.doRun(Build.java:164) 16:24:49 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:522) 16:24:49 at hudson.model.Run.execute(Run.java:1896) 16:24:49 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44) 16:24:49 at hudson.model.ResourceController.execute(ResourceController.java:101) 16:24:49 at hudson.model.Executor.run(Executor.java:442) 16:24:49 FATAL: Unable to delete script file /tmp/jenkins15763002140024398353.sh 16:24:49 java.io.EOFException 16:24:49 at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2905) 16:24:49 at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3400) 16:24:49 at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:936) 16:24:49 at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:379) 16:24:49 at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) 16:24:49 at hudson.remoting.Command.readFrom(Command.java:142) 16:24:49 at hudson.remoting.Command.readFrom(Command.java:128) 16:24:49 at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) 16:24:49 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61) 16:24:49 Caused: java.io.IOException: Unexpected termination of the channel 16:24:49 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75) 16:24:49 Caused: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@50af1e95:mhub-linux9": Remote call on mhub-linux9 failed. The channel is closing down or has closed down
I reviewed a number of the similarly named issues, but none of them were as recent as this LTS, with JDK-11.0.14