-
Bug
-
Resolution: Fixed
-
Major
-
Jenkins 2.249.2
Since the beginning of the year, we faced with Jenkins 2 master slowdowns : from GC logs, we suspect a memory leak ..
As you can see, the Old Gen memory increased by more than 800 MB in two weeks.
To confirm this memory leak suspicious : we made a heap dump.
The heap dump shows 51% of the old gen is used to store the attribute Hudson.model.hudson.labels which contains ConcurrentHashMap of 3517 labels used for each jobs launched from the starting of the jvm.
The ConcurrentHashMap containing labels doesn't appear to remove entries after the job has run.
This issue seems to be revealed since the workload on Jenkins has increased a lot in recent months.
500 Builds a day seems pretty normal usage, I don't see why there should be 3000 entries
The random string will not be kept permanently (sorry I was not aware of how the operation of the podTemplates worked) as the random string is not tied to a podTemplate directly but to a specific node, once the node is deleted it should be removed.
I'm curious what is actually in that particular Map (the actual values in the key / value field, especially the key)