Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46687

Job-DSL Leaks Memory Like A Sieve Thanks To Groovy

      Exactly what it says on the tin - unfortunately it's a stability problem for more intensive uses.  There was a previous fix done but more recent versions of Groovy from later cores no longer handle classes int he same way and the hacks to purge its memory-leaky ways need to change. 

      Now that we have gotten the "dangit groovy!" bits out of the way, there's an easy fix!  I would suggest incorporating the latest version of the fixes Pipeline is using to work around the issues from here: https://github.com/jenkinsci/workflow-cps-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L1052

      It should be a pretty much direct copy+paste, thankfully, and the above code is already pretty heavily tested (you can port over the tests too if'n y'like). 

          [JENKINS-46687] Job-DSL Leaks Memory Like A Sieve Thanks To Groovy

          Sam Gleske added a comment -

          svanoort in your example you recommend source code lines from the master branch which is a moving target; subject to change. I recommend picking a commit and then reference the source file and line from that commit so it stays the same.

          Sam Gleske added a comment - svanoort in your example you recommend source code lines from the master branch which is a moving target; subject to change. I recommend picking a commit and then reference the source file and line from that commit so it stays the same.

          Sam Gleske added a comment -

          Do you think this could be run from a Job DSL script? i.e. run a "clean up classloader" step at the end of running job dsl scripts?

          https://github.com/jenkinsci/workflow-cps-plugin/blob/d348018be09df2a742e6a4bdcb2efd2cf7a5ed41/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L1038-L1082

          Sam Gleske added a comment - Do you think this could be run from a Job DSL script? i.e. run a "clean up classloader" step at the end of running job dsl scripts? https://github.com/jenkinsci/workflow-cps-plugin/blob/d348018be09df2a742e6a4bdcb2efd2cf7a5ed41/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L1038-L1082

          Jesse Glick added a comment -

          I feel like this is a duplicate. There was some popular bug (in core?) about Groovy leaks but it did not apply to Pipeline. I cannot find it now.

          See JENKINS-23762 and JENKINS-29068 however.

          Jesse Glick added a comment - I feel like this is a duplicate. There was some popular bug (in core ?) about Groovy leaks but it did not apply to Pipeline. I cannot find it now. See  JENKINS-23762 and  JENKINS-29068 however.

          Sam Van Oort added a comment -

          jglick If I can get https://issues.jenkins-ci.org/browse/JENKINS-47758 working it might protect against this

          Sam Van Oort added a comment - jglick If I can get https://issues.jenkins-ci.org/browse/JENKINS-47758  working it might protect against this

          Sam Van Oort added a comment -

          https://issues.jenkins-ci.org/browse/JENKINS-47758 won't protect job-dsl because it has a very custom approach to parsing/classloading/executing the groovy – I might be able to add some hooks in script-security that job-dsl can consume to get the cleanup benefits though.  

          Sam Van Oort added a comment - https://issues.jenkins-ci.org/browse/JENKINS-47758  won't protect job-dsl because it has a very custom approach to parsing/classloading/executing the groovy – I might be able to add some hooks in script-security that job-dsl can consume to get the cleanup benefits though.  

            jamietanna Jamie Tanna
            svanoort Sam Van Oort
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: