-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Opensuse 11.2
We have a hudson instance that is failing after only a few builds with
the "Too Many Open file error". Here are the vitals:
Hudson 1.386
Git Plugin 1.1.4
The system is building a Rails test suite using the Rake plugin. Git
is configured to build all available branches and there are about 12
right now, but regular activity only occurs on 2. All builds are
being done on a slave machine using Unix SSH.
We used the instructions on the wiki related to the "too many open
files" error, ls -la /proc/PID/fd shows 2500 files and GC does not
clean them up. It essentially makes the slave useless until we
disconnect and reconnect to it.
If we don't have git configured to use all the branches then we don't
get the problem.
Here is a full backtrace The Git error at the top can vary
/usr/bin/git checkout -f 7e42dcecfc5a792e9865d01916cb1e8942jkfc35
FATAL: Could not checkout 7e42dcecfc5a792e9865d01916cb1e8942jkfc35
hudson.plugins.git.GitException: Could not checkout 7e42dcecfc5a792e9865d01916cb1e8942jkfc35
at hudson.plugins.git.GitAPI.checkout(GitAPI.java:450)
at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:981)
at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:972)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1899)
at hudson.remoting.UserRequest.perform(UserRequest.java:114)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: hudson.plugins.git.GitException: Error performing /usr/bin/git checkout -f 7e42dcecfc5a792e9865d01916cb1e8942jkfc35
Cannot run program "/usr/bin/git" (in directory "/home/web/workspace/MyProject"): java.io.IOException: error=24, Too many open files
at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:379)
at hudson.plugins.git.GitAPI.launchCommand(GitAPI.java:344)
at hudson.plugins.git.GitAPI.launchCommand(GitAPI.java:354)
at hudson.plugins.git.GitAPI.checkout(GitAPI.java:448)
... 12 more
Caused by: java.io.IOException: Cannot run program "/usr/bin/git" (in directory "/home/web/workspace/MyProject"): java.io.IOException: error=24, Too many open files
at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
at hudson.Proc$LocalProc.<init>(Proc.java:192)
at hudson.Proc$LocalProc.<init>(Proc.java:164)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:638)
at hudson.Launcher$ProcStarter.start(Launcher.java:273)
at hudson.Launcher$ProcStarter.join(Launcher.java:280)
at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:368)
... 15 more
Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)
at java.lang.ProcessImpl.start(ProcessImpl.java:81)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:468)
... 21 more