Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37984

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: General error during class generation: Method code too large! error in pipeline Script

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Hi,

      We are getting below error in Pipeline which has some 495 lines of groovy code. Initially we assumed that one of our methods has an issue but once we remove any 30-40 lines of Pipeline groovy, this issue is not coming.

      Can you please suggest a quick workaround. It's a blocker for us.

      org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
      General error during class generation: Method code too large!
      
      java.lang.RuntimeException: Method code too large!
      	at groovyjarjarasm.asm.MethodWriter.a(Unknown Source)
      	at groovyjarjarasm.asm.ClassWriter.toByteArray(Unknown Source)
      	at org.codehaus.groovy.control.CompilationUnit$16.call(CompilationUnit.java:815)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1053)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:591)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:67)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:410)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:373)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:213)
      	at hudson.model.ResourceController.execute(ResourceController.java:98)
      	at hudson.model.Executor.run(Executor.java:410)
      
      1 error
      
      	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1073)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:591)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:67)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:410)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:373)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:213)
      	at hudson.model.ResourceController.execute(ResourceController.java:98)
      	at hudson.model.Executor.run(Executor.java:410)
      Finished: FAILURE
      

        Attachments

          Issue Links

            Activity

            Hide
            bitwiseman Liam Newman added a comment - - edited

            Brian J Murrell
            There will always be a wall. The limitations in class size are hard coded into the Java Class file format.
            However, this improvement moves the the wall exponentially further out - similar to going from 16-bit integer to 32-bit integer. It is a massive improvement.

            Even if you are not encountering the issue currently, it would be helpful if you tried this new version to make sure it didn't break anything. Further, you could try reverting the last change that you made to your Jenkinsfile to mitigate this and see if it still works. The only change you might need to make is adding "@Field" to script local variable declarations (def varName="value" in the root of the script).

            Show
            bitwiseman Liam Newman added a comment - - edited Brian J Murrell There will always be a wall. The limitations in class size are hard coded into the Java Class file format. However, this improvement moves the the wall exponentially further out - similar to going from 16-bit integer to 32-bit integer. It is a massive improvement. Even if you are not encountering the issue currently, it would be helpful if you tried this new version to make sure it didn't break anything. Further, you could try reverting the last change that you made to your Jenkinsfile to mitigate this and see if it still works. The only change you might need to make is adding "@Field" to script local variable declarations (def varName="value" in the root of the script).
            Hide
            brianjmurrell Brian J Murrell added a comment -

            I don't have anything to revert.  I'd never commit a Jenkinsfile that doesn't run in Jenkins.  I wouldn't have the approvals to land such a patch.

            So the last time I ran into this, was when I added a stage or two but in the same commit I also refactored to allow the new stage(s) to fit.

            I'm also not sure when my priorities at my day job will allow me time to stand up a non-production Jenkins server to try this out in.  When I do find the time, I will be sure to update here.

            Show
            brianjmurrell Brian J Murrell added a comment - I don't have anything to revert.  I'd never commit a Jenkinsfile that doesn't run in Jenkins.  I wouldn't have the approvals to land such a patch. So the last time I ran into this, was when I added a stage or two but in the same commit I also refactored to allow the new stage(s) to fit. I'm also not sure when my priorities at my day job will allow me time to stand up a non-production Jenkins server to try this out in.  When I do find the time, I will be sure to update here.
            Hide
            mbrunton27 Matthew Brunton added a comment -

            I thought I'd add that I tested these changes with my skeleton script that reproduced the error for us and it seems to be working. I also can't make these changes to our main Jenkins instance, but I used my docker setup that I have for reproducing errors.

            Previously, I had narrowed down the cause for us to be the number of stages with "when" conditionals. When we get somewhere between 30 and 35 stages with "when" expressions, the error shows up, regardless of any other code in the pipeline (I was able to reproduce with a blank pipeline library with just echo lines).

            I installed the plugins and activated SCRIPT_SPLITTING_TRANSFORMATION, and now I've been able to run the same script with 60 stages without hitting the error. I might be able to go higher, but our use case is far from hitting that many stages.

            I do want to say thanks for keeping this issue active. We've been running a workaround script for a while now but I've been keeping my eye on progress on this issue, and it looks promising so far. I'm anxious to get back to a pure declarative implementation.

            Show
            mbrunton27 Matthew Brunton added a comment - I thought I'd add that I tested these changes with my skeleton script that reproduced the error for us and it seems to be working. I also can't make these changes to our main Jenkins instance, but I used my docker setup that I have for reproducing errors. Previously, I had narrowed down the cause for us to be the number of stages with "when" conditionals. When we get somewhere between 30 and 35 stages with "when" expressions, the error shows up, regardless of any other code in the pipeline (I was able to reproduce with a blank pipeline library with just echo lines). I installed the plugins and activated SCRIPT_SPLITTING_TRANSFORMATION, and now I've been able to run the same script with 60 stages without hitting the error. I might be able to go higher, but our use case is far from hitting that many stages. I do want to say thanks for keeping this issue active. We've been running a workaround script for a while now but I've been keeping my eye on progress on this issue, and it looks promising so far. I'm anxious to get back to a pure declarative implementation.
            Hide
            brianjmurrell Brian J Murrell added a comment -

            Previously, I had narrowed down the cause for us to be the number of stages with "when" conditionals. When we get somewhere between 30 and 35 stages with "when" expressions, the error shows up, regardless of any other code in the pipeline (I was able to reproduce with a blank pipeline library with just echo lines).

            By suspicion here is that the complexity of the when conditions adds to the amount of bytecode generated, contributing to the Method code too large situation. I moved all of my multi-condition tests into functions so that all of my when conditions are a single call to the function wrapping their actual multi-condition tests.

            I'm anxious to get back to a pure declarative implementation.

            Indeed. Without unnecessary indirection through functions that have no DRY purpose whatsoever and exist solely to reduce the size of the Method code.

            Show
            brianjmurrell Brian J Murrell added a comment - Previously, I had narrowed down the cause for us to be the number of stages with "when" conditionals. When we get somewhere between 30 and 35 stages with "when" expressions, the error shows up, regardless of any other code in the pipeline (I was able to reproduce with a blank pipeline library with just echo lines). By suspicion here is that the complexity of the when conditions adds to the amount of bytecode generated, contributing to the Method code too large situation. I moved all of my multi-condition tests into functions so that all of my when conditions are a single call to the function wrapping their actual multi-condition tests. I'm anxious to get back to a pure declarative implementation. Indeed. Without unnecessary indirection through functions that have no DRY purpose whatsoever and exist solely to reduce the size of the Method code .
            Hide
            doman18 Doman Panda added a comment - - edited

            I have couple questions about workarrounds:

            1. I saw that many recommend using sharing libraries. How it is different from using functions from the same file but outside of pipelines{} section? 
            2. Some also sugested me that separating functions in Jenkinsfile works only if you wrap arround pipeline{} section with with call() function like this - call(){pipeline{...}}. Is it true?
            3. Is it me or using matrix{} greatly rises the risk of getting such error? I mean it seems to me that i can have much larger pipelines when im not using them. Or is it because i use when{} more?
            4. Does things like number of variables, maps (arrays) or objects defined outside of pipeline script have an impact to this problem?
            5. Some say that using scripting (imperative) syntax does not trigger this problem. Ive never use it. Is it worth to learn it and introduce it in projects? 

            Im asking about those because I really hesitate to use share library solution. Most of my functions are not universal and doesnt make sense for any other projects. Also i use mutlibranch jobs a lot and cant imagine how static libs can work with dynamic branches when building process is strictly co-related with development process (Jenkinsfile changes with code development) and thus cant be separated. Change in code would have to reflect also in shared library. For example when developers add new compilation target, new matrix axis is being added to Jenkinsfile. And sometimes new section. How would this work in multibranch environment and shared library soultion where some branches work with new Jenkinsfile and some still have to be build old way ?

            Show
            doman18 Doman Panda added a comment - - edited I have couple questions about workarrounds: I saw that many recommend using sharing libraries. How it is different from using functions from the same file but outside of pipelines{} section?  Some also sugested me that separating functions in Jenkinsfile works only if you wrap arround pipeline{} section with with call() function like this - call(){pipeline{...}}. Is it true? Is it me or using matrix{} greatly rises the risk of getting such error? I mean it seems to me that i can have much larger pipelines when im not using them. Or is it because i use when{} more? Does things like number of variables, maps (arrays) or objects defined outside of pipeline script have an impact to this problem? Some say that using scripting (imperative) syntax does not trigger this problem. Ive never use it. Is it worth to learn it and introduce it in projects?  Im asking about those because I really hesitate to use share library solution. Most of my functions are not universal and doesnt make sense for any other projects. Also i use mutlibranch jobs a lot and cant imagine how static libs can work with dynamic branches when building process is strictly co-related with development process (Jenkinsfile changes with code development) and thus cant be separated. Change in code would have to reflect also in shared library. For example when developers add new compilation target, new matrix axis is being added to Jenkinsfile. And sometimes new section. How would this work in multibranch environment and shared library soultion where some branches work with new Jenkinsfile and some still have to be build old way ?

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              anudeeplalam Anudeep Lalam
              Votes:
              74 Vote for this issue
              Watchers:
              88 Start watching this issue

                Dates

                Created:
                Updated: