-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Jenkins 2.73.3
After upgrade to 2.73.3 we noticed that after a few days Jenkins becomes unresponsive and gives "too many open files" error. The only way to overcome the issue is to restart the instance.
The File Leak Detector Plugin does show some open handles for completed jobs but after a few hours those are pruned by themselves from the list. The total amount of open file handles never exceeds 2000 according to the plugin.
However when we run 'lsof' we see the increase of open file handles over time (see the screenshot attached: the big drop on Nov 27 is due to the manual Jenkins restart).
Here's the summary of 2 lsof runs with 1 day between them. The list of top files:
Nov-17:
100632 slave.log 32294 log 7685 timestamps 4193 random 3635 urandom
Nov-18:
708532 log 297707 timestamps 98280 slave.log 90675 Common.groovy 85995 BobHelper.groovy
Common.groovy and BobHelper.groovy are used by some jobs.
- is related to
-
-
- Resolved
-
[JENKINS-48280] File handle leak
Volodymyr Sobotovych
created issue -
Volodymyr Sobotovych
made changes -
| Attachment | New: Screen Shot 2017-11-29 at 9.45.10 AM.png [ 40525 ] |
Volodymyr Sobotovych
made changes -
| Attachment | Original: Screen Shot 2017-11-29 at 9.45.10 AM.png [ 40525 ] |
Volodymyr Sobotovych
made changes -
| Attachment | New: dpi-jenkins.png [ 40526 ] |
Volodymyr Sobotovych
made changes -
| Description |
Original:
After upgrade to 2.73.3 we noticed that after a few days Jenkins becomes unresponsive and gives "too many open files" error. The only way to overcome the issue is to restart the instance. The File Leak Detector Plugin does show some open handles for completed jobs but after a few hours those are pruned by themselves from the list. The total amount of open file handles never exceeds 2000 according to the plugin. However when we run 'lsof' we see the increase of open file handles over time (see the screenshot attached). Here's the summary of 2 lsof runs with 1 day between them. The list of top files: Nov-17: {code:java} 100632 slave.log 32294 log 7685 timestamps 4193 random 3635 urandom{code} Nov-18: {code} 708532 log 297707 timestamps 98280 slave.log 90675 Common.groovy 85995 BobHelper.groovy {code} Common.groovy and BobHelper.groovy are used by some jobs. |
New:
After upgrade to 2.73.3 we noticed that after a few days Jenkins becomes unresponsive and gives "too many open files" error. The only way to overcome the issue is to restart the instance. The File Leak Detector Plugin does show some open handles for completed jobs but after a few hours those are pruned by themselves from the list. The total amount of open file handles never exceeds 2000 according to the plugin. However when we run 'lsof' we see the increase of open file handles over time (see the screenshot attached: the big drop on Nov 27 is due to the manual Jenkins restart). Here's the summary of 2 lsof runs with 1 day between them. The list of top files: Nov-17: {code:java} 100632 slave.log 32294 log 7685 timestamps 4193 random 3635 urandom{code} Nov-18: {code} 708532 log 297707 timestamps 98280 slave.log 90675 Common.groovy 85995 BobHelper.groovy {code} Common.groovy and BobHelper.groovy are used by some jobs. |
Volodymyr Sobotovych
made changes -
| Link |
New:
This issue is blocked by |
Volodymyr Sobotovych
made changes -
| Link |
Original:
This issue is blocked by |
| Link |
New:
This issue is related to |