-
Bug
-
Resolution: Unresolved
-
Major
-
1.580.2
A Jenkins instance using this plugin was found to have more than 4400 threads with this stack trace:
"Waiting to acquire /…/workspace/… : Computer.threadPoolForRemoting [#37]" daemon prio=10 tid=0x… nid=0x… in Object.wait() [0x…] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:245) - locked <0x…> (a hudson.slaves.WorkspaceList) at hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:224) - locked <0x…> (a hudson.slaves.WorkspaceList) at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1446) at hudson.model.AbstractProject._poll(AbstractProject.java:1425) at hudson.model.AbstractProject.poll(AbstractProject.java:1336) at com.cloudbees.jenkins.GitHubPushTrigger$1.runPolling(GitHubPushTrigger.java:81) at com.cloudbees.jenkins.GitHubPushTrigger$1.run(GitHubPushTrigger.java:106) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
Looking at the logs, there were 1150 GitHub push notifications for this job over 12 hours. Each created a polling thread and competed for a lock on the workspace. Each polling request took 1-6 seconds.
Such a high thread count had other knock on effects, such as high CPU usage when counting threads.
So we should figure out a way to either eliminate the duplicate polling or just keep the thread count down.
- is related to
-
JENKINS-22456 github plugin too many open files unclosed github-polling.log filehandles causes all builds to fail
-
- Resolved
-
But in the current state when GH trigger calls GIT polling, that has custom algorithm for detecting new commits (that may never end triggering builds), gh plugin triggering probably should be merged per job.
Imho let's better migrate to commitNotify, WDYT?