Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68210

Call to any Step triggers Garbage Collector OutOfMemoryError

      I have two Jenkins servers. I updated a bunch of plugins on both yesterday. I believe a key one was Pipeline: Model APIVersion2.2075.vce74e77b_ce40

      After that all runs using agents launched by Amazon EC2 plugin
      Version1.68 would cause the error:

      java.lang.OutOfMemoryError: GC overhead limit exceeded

      Reverting from Pipeline Version2.7 to 2.6 and the error is no longer happening. No other changes were applied.

      Here's an example of the full error:

      java.lang.OutOfMemoryError: GC overhead limit exceeded
      Caused: java.io.IOException: Remote call on EC2 (AWS Dev Account) - Pacific Builder (i-00d8bd6e0c699d7dc) failed
      at hudson.remoting.Channel.call(Channel.java:1004)
      at hudson.FilePath.act(FilePath.java:1194)
      at hudson.FilePath.act(FilePath.java:1183)
      at hudson.FilePath.mkdirs(FilePath.java:1374)
      at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:97)
      at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
      at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE

      In the comments I'll list out the plugins and versions of the working system and the failing system

          [JENKINS-68210] Call to any Step triggers Garbage Collector OutOfMemoryError

          Geoff Dunn added a comment -

          Here's another one where I cut down the Jenkinsfile to fail on the step echo "Hello"
          java.lang.OutOfMemoryError: GC overhead limit exceeded
          Caused: java.io.IOException: Remote call on EC2 (AWS Dev Account) - Windows Test Executor (i-06614a52cb4ae8bb4) failed
          at hudson.remoting.Channel.call(Channel.java:1004)
          at hudson.FilePath.act(FilePath.java:1194)
          at hudson.FilePath.act(FilePath.java:1183)
          at hudson.FilePath.readToString(FilePath.java:2377)
          at org.jenkinsci.plugins.workflow.cps.steps.LoadStepExecution.start(LoadStepExecution.java:39)
          at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:322)
          at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
          at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
          at sun.reflect.GeneratedMethodAccessor1687.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
          at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
          at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
          at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
          at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
          at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
          at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:158)
          at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
          at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
          at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
          at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
          at WorkflowScript.run(WorkflowScript:10)
          at __cps.transform__(Native Method)
          at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
          at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
          at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
          at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
          at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
          at com.cloudbees.groovy.cps.Next.step(Next.java:83)
          at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
          at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
          at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
          at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
          at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
          at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
          at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
          at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294)
          at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
          at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE

          Geoff Dunn added a comment - Here's another one where I cut down the Jenkinsfile to fail on the step echo "Hello" java.lang.OutOfMemoryError: GC overhead limit exceeded Caused: java.io.IOException: Remote call on EC2 (AWS Dev Account) - Windows Test Executor (i-06614a52cb4ae8bb4) failed at hudson.remoting.Channel.call(Channel.java:1004) at hudson.FilePath.act(FilePath.java:1194) at hudson.FilePath.act(FilePath.java:1183) at hudson.FilePath.readToString(FilePath.java:2377) at org.jenkinsci.plugins.workflow.cps.steps.LoadStepExecution.start(LoadStepExecution.java:39) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:322) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124) at sun.reflect.GeneratedMethodAccessor1687.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:158) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:10) at __ cps.transform __(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83) at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE

          Geoff Dunn added a comment -

          Full plugin list is too much

          Both Working and not working systems are running Jenkins 2.332.1

          Key plugins on both

          Amazon EC2 plugin
          Version1.68

          Pipeline
          Version2.7

          Pipeline: API
          Version1143.v2d42f1e9dea_5

          Pipeline: Declarative
          Version2.2075.vce74e77b_ce40

          Pipeline: Groovy
          Version2683.vd0a_8f6a_1c263

          Pipeline: Job
          Version1174.vdcb_d054cf74a_

          Pipeline: Model API
          Version2.2075.vce74e77b_ce40

          Plugins only on the failing setup (even if not relevant):

          Amazon Web Services SDK :: Core
          Version1.12.69

          Amazon Web Services SDK :: JMespath
          Version1.12.69

          Amazon Web Services SDK :: KMS
          Version1.12.69

          Amazon Web Services SDK :: S3
          Version1.12.69

          Amazon Web Services SDK :: STS
          Version1.12.69

          Badge
          Version1.9.1

          Bitbucket Plugin
          Version223.vd12f2bca5430

          File Parameter Plugin
          Version191.v35851c9e7a_5a_

          Label Linked Jobs Plugin
          Version6.0.1

          Pipeline Graph View Plugin
          Version51.v5a693b766483

          Sidebar Link
          Version2.1.0

          Geoff Dunn added a comment - Full plugin list is too much Both Working and not working systems are running Jenkins 2.332.1 Key plugins on both Amazon EC2 plugin Version1.68 Pipeline Version2.7 Pipeline: API Version1143.v2d42f1e9dea_5 Pipeline: Declarative Version2.2075.vce74e77b_ce40 Pipeline: Groovy Version2683.vd0a_8f6a_1c263 Pipeline: Job Version1174.vdcb_d054cf74a_ Pipeline: Model API Version2.2075.vce74e77b_ce40 Plugins only on the failing setup (even if not relevant): Amazon Web Services SDK :: Core Version1.12.69 Amazon Web Services SDK :: JMespath Version1.12.69 Amazon Web Services SDK :: KMS Version1.12.69 Amazon Web Services SDK :: S3 Version1.12.69 Amazon Web Services SDK :: STS Version1.12.69 Badge Version1.9.1 Bitbucket Plugin Version223.vd12f2bca5430 File Parameter Plugin Version191.v35851c9e7a_5a_ Label Linked Jobs Plugin Version6.0.1 Pipeline Graph View Plugin Version51.v5a693b766483 Sidebar Link Version2.1.0

            Unassigned Unassigned
            geoff Geoff Dunn
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: