-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Jenkins ver. 2.150.2
There is partial fix for this in pipeline-model-definition-plugin v1.4.0 and later, significantly improved in v1.8.2. Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting a JVM property (either on the command-line or in Jenkins script console):
org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true
As noted, this still works best with a Jenkinsfile with pipeline directive as the only root item in the file.
Since v1.8.2 this workaround reports an informative error for pipelines using `def` variables before the pipeline directive. Add a @Field annotation to those declaration.
This workaround generally does NOT work if the pipeline directive inside a shared library method. If this is a scenario you want, please come join the pipeline authoring SIG and we can discuss.Please give it a try and provide feedback.
When working with a declarative pipeline script, we run into an error "Method Code too large".
This seems to happen when the code between 'pipeline{}' is more than a specific size.
I'm creating this issue for this component following the suggestion of Jesse Glick at the issue 37984
I've attached a declarative pipeline script that reproduces the issue.
- is related to
-
-
- Open
-
[JENKINS-56500] Declarative pipeline restricted in code size
Hi mbost. I am still unable to get it to work even though I ran the script in Script Console. Adding "-vmargs" before the JVM parameter renders Jenkins service not startable at all.
Just curious, my project type is a multibranch pipeline project. Is yours a multibranch pipeline project as well? Perhaps if yours is not, that might explain why it is not working for me.
Marko
added a comment - Hi allancth.
No, it's not that, I'm also using a Multibranch pipeline project.
The next thing I would've had to try, had it not worked, would've been to extract some pieces into a shared library, as much as I despise the complexity of it (it having to be a whole new repo, not a file in my current project), so you might try the same.
Marko
added a comment - Hi allancth .
No, it's not that, I'm also using a Multibranch pipeline project.
The next thing I would've had to try, had it not worked, would've been to extract some pieces into a shared library, as much as I despise the complexity of it (it having to be a whole new repo, not a file in my current project), so you might try the same. Thanks mbost for the response.
The workaround I have now is to split the MethodTooLarge Jenkinsfile into several separate jobs. Actually I think for scalability I need to model the build jobs to the individual modules but that's too much to handle right now.
I will try to get the some of the pieces into a shared library but for a quick workaround I can also move some into a shell script. I need to find out the benefits of putting some parts of the workflow into a shared library - what can be done in the shared library that a shell script can't.
Cristian
added a comment - SCRIPT_SPLITTING_TRANSFORMATION helps, but it seems to me that there is additionally a specific problem with the matrix step. For the second time, I have been able to solve the issue in one of my projects by changing a matrix step into a script step using the parallel step.
We build for tens of platforms, so it arguably an unusually big matrix. But there is only one axis, and it's tens of targets, not hundreds. The code inside the matrix doesn't even need to be specially complicated, it's mainly the amount of parallel branches that triggers the issue for us.
Cristian
added a comment - SCRIPT_SPLITTING_TRANSFORMATION helps, but it seems to me that there is additionally a specific problem with the matrix step. For the second time, I have been able to solve the issue in one of my projects by changing a matrix step into a script step using the parallel step.
We build for tens of platforms, so it arguably an unusually big matrix. But there is only one axis, and it's tens of targets, not hundreds. The code inside the matrix doesn't even need to be specially complicated, it's mainly the amount of parallel branches that triggers the issue for us. FYI, we worked around this issue long ago by adding a scripted step inside a declarative pipeline.
This looks like this:
stage('Testing') { steps { script { def testers = getParallelTesters() // returns *hundreds* of parallel stages parallel testers } } post { // ... } }
where
def getParallelTesters() {
def testers = [:].asSynchronized()
testers["Test 1"] = { return test1() }
// ....
testers["Test 999"] = { return test999() }
return testers
}
where smaller functions return a closure wrapping stage and node as needed.
This gives us the same huge pipeline while limiting the size of declarative pipeline to a minimum.
efo plo
added a comment - - edited
FYI, we worked around this issue long ago by adding a scripted step inside a declarative pipeline.
This looks like this:
stage( 'Testing' ) {
steps {
script {
def testers = getParallelTesters() // returns *hundreds* of parallel stages
parallel testers
}
}
post {
// ...
}
}
where
def getParallelTesters() {
def testers = [:].asSynchronized()
testers[ "Test 1" ] = { return test1() }
// ....
testers[ "Test 999" ] = { return test999() }
return testers
}
where smaller functions return a closure wrapping stage and node as needed.
This gives us the same huge pipeline while limiting the size of declarative pipeline to a minimum. Cristian
added a comment - Yes, I did the same.
But that looks like the obvious way a matrix step would be implemented, no? Which raises the question of why the matrix step generates a "Method too large" to start with.
Cristian
added a comment - Yes, I did the same.
But that looks like the obvious way a matrix step would be implemented, no? Which raises the question of why the matrix step generates a "Method too large" to start with. 
Aleksandr
added a comment - Good day, colleagues!
We have the same issue, and I could confirm that it is related to the number of values in the matrix.
Currently we have the next one
axes {
axis {
name 'geo'
values 'pl', 'ee', 'lt', 'lv', 'sa', 'ua', 'bg', 'ro', 'hr', 'gr', 'cy', 'rs', 'kz', 'si', 'ba', 'hu'
}
}
But when we try to add one more value we are getting the next error:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during class generation: Method too large: WorkflowScript.___cps___156545 ()Lcom/cloudbees/groovy/cps/impl/CpsFunction;
groovyjarjarasm.asm.MethodTooLargeException: Method too large: WorkflowScript.___cps___156545 ()Lcom/cloudbees/groovy/cps/impl/CpsFunction;
at groovyjarjarasm.asm.MethodWriter.computeMethodInfoSize(MethodWriter.java:2087)
at groovyjarjarasm.asm.ClassWriter.toByteArray(ClassWriter.java:447)
at org.codehaus.groovy.control.CompilationUnit$17.call(CompilationUnit.java:850)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1087)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:190)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:175)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:568)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:518)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:336)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1107)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:190)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:175)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:568)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:518)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:336)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Stages block is very simple, even few one line steps.
Aleksandr
added a comment - Good day, colleagues!
We have the same issue, and I could confirm that it is related to the number of values in the matrix.
Currently we have the next one
axes {
axis {
name 'geo'
values 'pl' , 'ee' , 'lt' , 'lv' , 'sa' , 'ua' , 'bg' , 'ro' , 'hr' , 'gr' , 'cy' , 'rs' , 'kz' , 'si' , 'ba' , 'hu'
}
}
But when we try to add one more value we are getting the next error:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during class generation: Method too large: WorkflowScript.___cps___156545 ()Lcom/cloudbees/groovy/cps/impl/CpsFunction;
groovyjarjarasm.asm.MethodTooLargeException: Method too large: WorkflowScript.___cps___156545 ()Lcom/cloudbees/groovy/cps/impl/CpsFunction;
at groovyjarjarasm.asm.MethodWriter.computeMethodInfoSize(MethodWriter.java:2087)
at groovyjarjarasm.asm.ClassWriter.toByteArray(ClassWriter.java:447)
at org.codehaus.groovy.control.CompilationUnit$17.call(CompilationUnit.java:850)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1087)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:190)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:175)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:568)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:518)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:336)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1107)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:190)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:175)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:568)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:518)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:336)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Stages block is very simple, even few one line steps. 
John Malmberg
added a comment - I would recommend using Jekinsfile2 instead of Jenkinsfile when this setting is active so that a project can support the default Jenkins setup while migrating to this setup for larger pipelines.
John Malmberg
added a comment - I would recommend using Jekinsfile2 instead of Jenkinsfile when this setting is active so that a project can support the default Jenkins setup while migrating to this setup for larger pipelines. John Malmberg
added a comment - This is a problem:
@Library(value='pipeline-lib@your_branch') _
Causes:
17:25:43 org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: 17:25:43 General error during semantic analysis: [JENKINS-37984] SCRIPT_SPLITTING_TRANSFORMATION is an experimental feature of Declarative Pipeline and is incompatible with local variable declarations inside a Jenkinsfile. As a temporary workaround, you can add the '@Field' annotation to these local variable declarations. However, use of Groovy variables in Declarative pipeline, with or without the '@Field' annotation, is not recommended or supported. To use less effective script splitting which allows local variable declarations without changing your pipeline code, set SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES=true . Local variable declarations found: _.
John Malmberg
added a comment - This is a problem:
@Library(value= 'pipeline-lib@your_branch' ) _
Causes:
17:25:43 org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
17:25:43 General error during semantic analysis: [JENKINS-37984] SCRIPT_SPLITTING_TRANSFORMATION is an experimental feature of Declarative Pipeline and is incompatible with local variable declarations inside a Jenkinsfile. As a temporary workaround, you can add the '@Field' annotation to these local variable declarations. However, use of Groovy variables in Declarative pipeline, with or without the '@Field' annotation, is not recommended or supported. To use less effective script splitting which allows local variable declarations without changing your pipeline code, set SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES= true . Local variable declarations found: _.
OK, I'm reporting back my experience converting a scripted pipeline (900 LoC) into a declarative one (1k LoC).
First, let me just say that I love the neat structure and the whole experience of working with declarative pipelines. The conversion felt like tidying up.
I hit the issue when I added the last two stages.
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: General error during class generation: Method too large: WorkflowScript.__cps__9839 ()Lcom/cloudbees/groovy/cps/impl/CpsFunction;But after adding
org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=trueto my Manage Jenkins > Script Console (just paste the snippet above and click `Run` and then start a new build)
the issue went away.
Just for the sake of completeness, I also had to add the same setting to `javaOpts` in my Jenkins' helm chart (I'm using k8s + JCasC), so I don't have to manually set it in the Script Console on all newly installed Jenkins instances:
javaOpts: "...other options -Dorg.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true"