[JENKINS-33358] Groovy and PermGen memory leak - Jenkins Jira

Hongkai Liu created issue - 2016-03-07 18:07

Hongkai Liu made changes - 2016-03-07 18:09

Field	Original Value	New Value
Description	Configure 10 jobs which build per minute Each build triggers a groovy script by Global Post Script Plugin. {noformat} class Main { static void main(String... args) { println 'Groovy world!' def p = new Person() p.write() } } class Person { def write(){ println 'i am writing ...!' } } {noformat} After sometime, the perm gen is eaten up. See the screenshot of jvisualvm. I checked the source code of the plugin: https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder 1. if this is a potential memory leak, and 2. if anyone has met similar issue.	Configure 10 jobs which build per minute Each build triggers a groovy script by Global Post Script Plugin. {noformat} class Main { static void main(String... args) { println 'Groovy world!' def p = new Person() p.write() } } class Person { def write(){ println 'i am writing ...!' } } {noformat} After sometime, the perm gen is eaten up. See the screenshot of jvisualvm. !Screenshot from 2016-03-07 09.43.29.png\|thumbnail! I checked the source code of the plugin: https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder 1. if this is a potential memory leak, and 2. if anyone has met similar issue.

Hongkai Liu made changes - 2016-03-07 18:11

Issue Type

Bug [ 1 ]

Story [ 10002 ]

Hongkai Liu made changes - 2016-03-07 18:54

Labels

groovy memory-leak

Hongkai Liu made changes - 2016-03-07 19:23

Description

Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

We got the problem from the operation team who observe that the perm gen size has been increased by monitoring plugin.
This is quite important for us because more and more groovy based steps are integrated into our CI procedure.
After analysis, we simplify the testing environment with 10 jobs and Global Post Script Plugin.

We test further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

Hongkai Liu made changes - 2016-03-07 19:28

Description

We got the problem from the operation team who observe that the perm gen size has been increased by monitoring plugin.
This is quite important for us because more and more groovy based steps are integrated into our CI procedure.
After analysis, we simplify the testing environment with 10 jobs and Global Post Script Plugin.

We test further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

We got the problem from the operation team who observe that the perm gen size has been increased by monitoring plugin.
This is quite important for us because more and more groovy based steps are integrated into our CI procedure.
After analysis, we simplify the testing environment with 10 jobs and Global Post Script Plugin.

We test further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there.
<dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>2.4.6</version>
</dependency>

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

Hongkai Liu made changes - 2016-03-07 19:28

Priority

Major [ 3 ]

Critical [ 2 ]

Christian Lague made changes - 2016-03-07 19:35

Issue Type

Story [ 10002 ]

Bug [ 1 ]

Christian Lague made changes - 2016-03-07 19:35

Component/s		core [ 15593 ]
Component/s	global-post-script-plugin [ 20637 ]

Christian Lague made changes - 2016-03-07 19:41

Assignee

Hao CHEN [ orctom ]

Jesse Glick [ jglick ]

Christian Lague made changes - 2016-03-10 17:58

Description

We got the problem from the operation team who observe that the perm gen size has been increased by monitoring plugin.
This is quite important for us because more and more groovy based steps are integrated into our CI procedure.
After analysis, we simplify the testing environment with 10 jobs and Global Post Script Plugin.

We test further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there.
<dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>2.4.6</version>
</dependency>

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

We have recently experienced OutOfMemory errors on some large Jenkins instances that predominantly have frequently executed jobs that contain either System Groovy scripts or Build Flow project types. In one case after disabling a groovy script that was being executed via the Global Post Script Plugin after every build, the PermGen profile improved as shown:

We test further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there.
<dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>2.4.6</version>
</dependency>

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

Christian Lague made changes - 2016-03-10 18:00

Description

We have recently experienced OutOfMemory errors on some large Jenkins instances that predominantly have frequently executed jobs that contain either System Groovy scripts or Build Flow project types. In one case after disabling a groovy script that was being executed via the Global Post Script Plugin after every build, the PermGen profile improved as shown:

We test further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there.
<dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>2.4.6</version>
</dependency>

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

We have recently experienced OutOfMemory errors on some large Jenkins instances that predominantly have frequently executed jobs that contain either System Groovy scripts or Build Flow project types. In one case after disabling a groovy script that was being executed via the Global Post Script Plugin after every build, the PermGen profile improved dramatically.

We tested further with GroovyShell to run a groovy script, not depending on Jenkins.
Java7: java.lang.OutOfMemoryError: PermGen space
Java8: java.lang.OutOfMemoryError: Java heap space

The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there.
<dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>2.4.6</version>
</dependency>

---------------------------------------
Configure 10 jobs which build per minute
Each build triggers a groovy script by Global Post Script Plugin.

{noformat}
class Main {
    static void main(String... args) {
        println 'Groovy world!'
        def p = new Person()
        p.write()
    }
}

class Person {
    def write(){
        println 'i am writing ...!'
    }
}
{noformat}

After sometime, the perm gen is eaten up. See the screenshot of jvisualvm.
!Screenshot from 2016-03-07 09.43.29.png|thumbnail!

I checked the source code of the plugin:
https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java

My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder
1. if this is a potential memory leak, and
2. if anyone has met similar issue.

Christian Lague made changes - 2016-03-10 18:04

Attachment		permgenIssue.jpg [ 32133 ]
Description	We have recently experienced OutOfMemory errors on some large Jenkins instances that predominantly have frequently executed jobs that contain either System Groovy scripts or Build Flow project types. In one case after disabling a groovy script that was being executed via the Global Post Script Plugin after every build, the PermGen profile improved dramatically. We tested further with GroovyShell to run a groovy script, not depending on Jenkins. Java7: java.lang.OutOfMemoryError: PermGen space Java8: java.lang.OutOfMemoryError: Java heap space The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there. <dependency> <groupId>org.codehaus.groovy</groupId> <artifactId>groovy-all</artifactId> <version>2.4.6</version> </dependency> --------------------------------------- Configure 10 jobs which build per minute Each build triggers a groovy script by Global Post Script Plugin. {noformat} class Main { static void main(String... args) { println 'Groovy world!' def p = new Person() p.write() } } class Person { def write(){ println 'i am writing ...!' } } {noformat} After sometime, the perm gen is eaten up. See the screenshot of jvisualvm. !Screenshot from 2016-03-07 09.43.29.png\|thumbnail! I checked the source code of the plugin: https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder 1. if this is a potential memory leak, and 2. if anyone has met similar issue.	We have recently experienced OutOfMemory errors on some large Jenkins instances that predominantly have frequently executed jobs that contain either System Groovy scripts or Build Flow project types. In one case after disabling a groovy script that was being executed via the Global Post Script Plugin after every build, the PermGen profile improved dramatically. !permgenIssue.jpg\|thumbnail! We tested further with GroovyShell to run a groovy script, not depending on Jenkins. Java7: java.lang.OutOfMemoryError: PermGen space Java8: java.lang.OutOfMemoryError: Java heap space The tests use 2.4.6 version of groovy which is the latest version. However, the issue is still there. <dependency> <groupId>org.codehaus.groovy</groupId> <artifactId>groovy-all</artifactId> <version>2.4.6</version> </dependency> --------------------------------------- Configure 10 jobs which build per minute Each build triggers a groovy script by Global Post Script Plugin. {noformat} class Main { static void main(String... args) { println 'Groovy world!' def p = new Person() p.write() } } class Person { def write(){ println 'i am writing ...!' } } {noformat} After sometime, the perm gen is eaten up. See the screenshot of jvisualvm. !Screenshot from 2016-03-07 09.43.29.png\|thumbnail! I checked the source code of the plugin: https://github.com/jenkinsci/global-post-script-plugin/blob/master/src/main/java/com/orctom/jenkins/plugin/globalpostscript/ScriptExecutor.java My feeling is that it is probably caused by GroovyShell. Jenkins loads and runs groovy extensively and maybe there are places in core or other plugins having similar logic, I wonder 1. if this is a potential memory leak, and 2. if anyone has met similar issue.

Hao CHEN made changes - 2016-03-23 01:52

Assignee

Jesse Glick [ jglick ]

Hongkai Liu [ hongkailiu ]

Hongkai Liu made changes - 2016-04-01 15:07

Attachment

Screenshot from 2016-04-01 11.04.43.png [ 32347 ]

Hongkai Liu made changes - 2016-04-01 15:16

Assignee

Hongkai Liu [ hongkailiu ]

Hao CHEN [ orctom ]

R. Tyler Croy made changes - 2016-07-25 23:52

Workflow

JNJira [ 169271 ]

JNJira + In-Review [ 183427 ]

Daniel Spilker made changes - 2016-08-31 06:49

Link

This issue is related to ~~JENKINS-37213~~ [ ~~JENKINS-37213~~ ]

GMC Software Development B&R Corporate made changes - 2016-09-26 06:01

Link

This issue relates to ~~JENKINS-38334~~ [ ~~JENKINS-38334~~ ]

Daniel Beck made changes - 2016-09-29 11:24

Link

This issue is duplicated by ~~JENKINS-38503~~ [ ~~JENKINS-38503~~ ]

Wisen Tanasa made changes - 2016-11-14 10:45

Attachment

before.png [ 34830 ]

Wisen Tanasa made changes - 2016-11-14 10:45

Attachment

after.png [ 34831 ]

SCM/JIRA link daemon made changes - 2017-02-17 23:26

Resolution		Fixed [ 1 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Jesse Glick made changes - 2017-02-22 22:38

Link

This issue is blocked by ~~JENKINS-42189~~ [ ~~JENKINS-42189~~ ]

Daniel Beck made changes - 2017-03-03 11:11

Labels

groovy memory-leak

groovy lts-candidate memory-leak

Oleg Nenashev made changes - 2017-03-10 07:21

Link

This issue is related to ~~JENKINS-42637~~ [ ~~JENKINS-42637~~ ]

Oliver Gondža made changes - 2017-03-15 11:01

Labels

groovy lts-candidate memory-leak

2.46.1-rejected groovy lts-candidate memory-leak

paul young made changes - 2017-03-28 03:37

Assignee

Hao CHEN [ orctom ]

paul young [ ympaul2 ]

Oliver Gondža made changes - 2017-04-13 12:02

Labels

2.46.1-rejected groovy lts-candidate memory-leak

2.46.1-rejected 2.46.2-rejected groovy lts-candidate memory-leak

Oliver Gondža made changes - 2017-05-02 13:52

Labels

2.46.1-rejected 2.46.2-rejected groovy lts-candidate memory-leak

2.46.1-rejected 2.46.2-rejected 2.46.3-rejected groovy memory-leak

James Dumay made changes - 2017-12-04 04:54

Remote Link

This issue links to "CloudBees Internal OSS-2127 (Web Link)" [ 18410 ]

James Dumay made changes - 2017-12-04 04:56

Remote Link

This issue links to "CloudBees Internal OSS-2079 (Web Link)" [ 18429 ]

Derek Kinney made changes - 2018-02-07 16:29

Link

This issue relates to ~~JENKINS-38334~~ [ ~~JENKINS-38334~~ ]

Derek Kinney made changes - 2018-02-07 16:30

Link

This issue relates to ~~JENKINS-38334~~ [ ~~JENKINS-38334~~ ]

Jenkins

Groovy and PermGen memory leak

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates