-
Bug
-
Resolution: Unresolved
-
Critical
-
None
This is to track the problem originally reported here: http://n4.nabble.com/Polling-hung-td1310838.html#a1310838
The referenced thread is relocated to http://jenkins.361315.n4.nabble.com/Polling-hung-td1310838.html
What the problem boils down to is that many remote operations are performed synchronously causing the channel object to be locked while a response returns. In situations where a lengthy remote operations is using the channel, SCM polling can be blocked waiting for the monitor on the channel to be released. In extreme situations, all the polling threads can wind up waiting on object monitors for the channel objects, preventing further processing of polling tasks.
Furthermore, if the slave dies, the locked channel object still exists in the master JVM. If no IOException is thrown to indicate the termination of the connection to the pipe, the channel can never be closed because Channel.close() itself is a sychronized operation.
- is related to
-
JENKINS-5760 Locked/hanged remote channel causing freezed job and lots of blocked threads.
-
- Resolved
-
-
JENKINS-12302 Remote call on CLI channel from [ip] failed
-
- Closed
-
-
JENKINS-19055 In case of connection loss, slave JVM should restart itself if it can
-
- Resolved
-
- links to
02:36:16 Started by upstream project "echidna-patch-quality" build number 335
02:36:16 originally caused by:
02:36:16 Started by command line by xxx
02:36:16 [EnvInject] - Loading node environment variables.
02:36:17 Building remotely on ECHIDNA-QUALITY (6.1 windows-6.1 windows amd64-windows amd64-windows-6.1 amd64) in workspace c:\buildfarm-slave\workspace\echidna-patch-compile
02:36:18 > git rev-parse --is-inside-work-tree
02:36:19 Fetching changes from the remote Git repository
02:36:19 > git config remote.origin.url ssh://*@...:*/ghts/ta
02:36:20 Fetching upstream changes from ssh://*@...:*/ghts/ta
02:36:20 > git --version
02:36:20 > git fetch --tags --progress ssh://*@...:/ghts/ta +refs/heads/:refs/remotes/origin/*
02:56:20 ERROR: Timeout after 20 minutes
02:56:20 FATAL: Failed to fetch from ssh://*@...:*/ghts/ta
02:56:20 hudson.plugins.git.GitException: Failed to fetch from ssh://bmcdiags@10.110.61.117:30000/ghts/ta
02:56:20 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:623)
02:56:20 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:855)
02:56:20 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:880)
02:56:20 at hudson.model.AbstractProject.checkout(AbstractProject.java:1414)
02:56:20 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:671)
02:56:20 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
02:56:20 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:580)
02:56:20 at hudson.model.Run.execute(Run.java:1684)
02:56:20 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
02:56:20 at hudson.model.ResourceController.execute(ResourceController.java:88)
02:56:20 at hudson.model.Executor.run(Executor.java:231)
02:56:20 Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress ssh://*@...:/ghts/ta +refs/heads/:refs/remotes/origin/*" returned status code -1:
02:56:20 stdout:
02:56:20 stderr: Could not create directory 'c/Users/Administrator/.ssh'.
02:56:20
02:56:20 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1325)
02:56:20 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1186)
02:56:20 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:87)
02:56:20 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:257)
02:56:20 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
02:56:20 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
02:56:20 at hudson.remoting.UserRequest.perform(UserRequest.java:118)
02:56:20 at hudson.remoting.UserRequest.perform(UserRequest.java:48)
02:56:20 at hudson.remoting.Request$2.run(Request.java:326)
02:56:20 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
02:56:20 at java.util.concurrent.FutureTask.run(Unknown Source)
02:56:20 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
02:56:20 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
02:56:20 at hudson.remoting.Engine$1$1.run(Engine.java:63)
02:56:20 at java.lang.Thread.run(Unknown Source)