(Thought this was already filed, but cannot find it now.)

There should be a component which records heap histograms using jmap -histo:live or something equivalent in-process. This can offer at least a guess as to the cause of increasing memory usage, pending an actual heap dump to pinpoint the problem.