Since upgrading a large number (100s) of Jenkins instances to LTS v2.277.2, and the associated plugin updates required for the UI changes, I have observed a number of these instances are leak memory, and in one case it only lasts about 36 hours before exhausting the heap (in that case it is 6GB).

a heapdump (e.g. jmap -dump:live,file=jenkins-20210603-0939.hprof) showed that there were a large number (1000s) of hudson.remoting.Channel classes retained in a list by the class org.jvnet.hudson.test.ChannelShutdownListener. Doing a grep in the installed plugins folder, the only plugin that referenced this class was the gerrit-trigger plugin:

plugins$ grep -iRH "org.jvnet.hudson.test.ChannelShutdownListener" *
Binary file gerrit-trigger/WEB-INF/lib/jenkins-test-harness-2.56.jar matches
 
I found it curious that a test class was being included in a production plugin, so I do not know why this jenkins-test-harness is bundled into the plugin.

Uninstalling (or disabling) the gerrit-trigger plugin (and in some cases the sonar-gerrit plugin as well, which has gerrit-trigger as a dependency) and restarting caused this ChannelShutdownListener to no longer be loaded (as evidenced by the output of jmap -histo:live <pid> | grep org.jvnet.hudson.test.ChannelShutdownListener), and the memory profile has remained flat over the past 24 hours.

I have attached a screenshot of the leak suspect from MAT https://www.eclipse.org/mat/
 
The jenkins-test-harness is pulled in here in the pom: https://github.com/jenkinsci/gerrit-trigger-plugin/blob/master/pom.xml#L280-L283, which was brought in with this commit in February: https://github.com/jenkinsci/gerrit-trigger-plugin/commit/04fd6aff727c0104344c91c72dda372c23cac4ba

The release notes suggest that 2.34.0 was the release that brought in in the change as part of https://issues.jenkins.io/browse/JENKINS-53250

I don't understand the interaction of this test-harness library and the running system, but it seems to be injecting itself in and causing the leak. If it's just a test harness it should probably be a dev dependency and not included in the main plugin, perhaps it came in by mistake?
 
Note: It does not appear to matter whether the gerrit plugin in used or not (and in the worst case I mentioned before, it wasn't used at all), so we are working to uninstall it where it isn't needed.