-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
Jenkins version 2.176.2 LTS
OpenShift 3.9 cluster
At the end of a build, it seems to hang for several minutes while Xvnc is shutting down ("Terminating xvnc.").
Several observations:
- This only happens on dynamic agents running on our OpenShift cluster. It does not happen on static/dedicated build agents or when running on a Jenkins master.
- It takes ~6 minutes to terminate when the option "Create a dedicated Xauthority file per build" is enabled. It takes "only" ~2 minutes when this option is disabled. This was reproducible across several builds switching the option on and off.
- I was able to narrow down the problem to this line in the code:
launcher.kill(xauthorityEnv);
- Since it involves core functionality (Launcher, ProcessTree, etc) I also added "core" as component.
- Please find thread dumps below:
"pool-1-thread-16 for JNLP4-connect connection to <cluster-hostname>/<cluster-IP>:50000 id=9736" Id=31 Group=main TIMED_WAITING at java.lang.Thread.sleep(Native Method) at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:784) at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:818) at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:805) at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:709) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1103) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1094) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at hudson.remoting.Engine$1$$Lambda$3/2070832539.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@195341cb
"Executor #0 for migration-agent-tvwtb : executing xvnc-terminating-test #16 / waiting for JNLP4-connect connection from <agent-IP>/<agent-IP>:35020 id=9736" Id=34283 Group=main TIMED_WAITING on hudson.remoting.UserRequest@480bc704 (in native) at java.lang.Object.wait(Native Method) - waiting on hudson.remoting.UserRequest@480bc704 at java.lang.Object.wait(Unknown Source) at hudson.remoting.Request.call(Request.java:177) - locked hudson.remoting.UserRequest@480bc704 at hudson.remoting.Channel.call(Channel.java:956) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1086) at hudson.plugins.xvnc.Xvnc$DisposerImpl.tearDown(Xvnc.java:248) at jenkins.tasks.SimpleBuildWrapper$EnvironmentWrapper.tearDown(SimpleBuildWrapper.java:175) at hudson.model.Build$BuildExecution.doRun(Build.java:174) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504) at hudson.model.Run.execute(Run.java:1818) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429)
