Details
-
Bug
-
Status: Reopened (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
-
Jenkins ver. 2.150.2
Description
There is partial fix for this in pipeline-model-definition-plugin v1.4.0 and later, significantly improved in v1.8.2. Due to the extent to which it change how pipelines are executed it is turned off by default. It can be turned on by setting a JVM property (either on the command-line or in Jenkins script console):
org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true
As noted, this still works best with a Jenkinsfile with pipeline directive as the only root item in the file.
Since v1.8.2 this workaround reports an informative error for pipelines using `def` variables before the pipeline directive. Add a @Field annotation to those declaration.
This workaround generally does NOT work if the pipeline directive inside a shared library method. If this is a scenario you want, please come join the pipeline authoring SIG and we can discuss.Please give it a try and provide feedback.
When working with a declarative pipeline script, we run into an error "Method Code too large".
This seems to happen when the code between 'pipeline{}' is more than a specific size.
I'm creating this issue for this component following the suggestion of Jesse Glick at the issue 37984
I've attached a declarative pipeline script that reproduces the issue.
Attachments
Issue Links
- is related to
-
JENKINS-37984 org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: General error during class generation: Method code too large! error in pipeline Script
-
- Open
-
Activity
oleksazhel
Interesting. This feature is still experimental, but I'm surprised that would be an issue. Do you mean Jenkins job parameters or do you mean method parameters into the shared library method? Perhaps you could open a new issue and link it to this one?
bitwiseman Yes, the issue was with passing parameters from method into method with pipeline inside. But I made workaround by code refactoring from fully declarative to hybrid where declarative steps call scripted pipeline methods.
Good day. bitwiseman I would like to start by thanking you for the work on this problem. I have a couple questions:
- Is this problem related or the same one whereby due to Jenkins use of CPS the bytecode size for a method is limited to 64K?
- Can you foresee this problem to ever be completely fixed? I would be willing to help if you think that is possible.
Thanks al lot
I get the problem with this (simplified) pipeline
import groovy.transform.Field def docker_arguments(boolean with_source_package) { if(with_source_package) { return "--network none -v ${env.WORKSPACE}/src:${env.WORKSPACE}/src:ro,z" } else { return "--network none -v ${env.WORKSPACE}/src:${env.WORKSPACE}/src:ro,z" } } @Field def base_image_version = "11" @Field def toolchain = [ a: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], b: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], c: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], d: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: true, other_var: false], e: [version: "8", cpack_gen: "RPM", source_package: false, other_var: false], f: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], g: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], h: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], i: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], j: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: true, other_var: false], k: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: true, other_var: false], l: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], m: [version: "${base_image_version}.2", cpack_gen: "TGZ", source_package: false, other_var: false], n: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], o: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], p: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], q: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], r: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], s: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], t: [version: "${base_image_version}.2", cpack_gen: "TGZ", source_package: false, other_var: false], u: [version: "${base_image_version}.2", cpack_gen: "TGZ", source_package: false, other_var: false], v: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], w: [version: "1", cpack_gen: "DEB", source_package: false, other_var: false], x: [version: "6", cpack_gen: "DEB", source_package: false, other_var: false], y: [version: "1", cpack_gen: "DEB", source_package: false, other_var: false], z: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], aa: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], ab: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], ac: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: true], ad: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 1: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 2: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 3: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 4: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 5: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 6: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 7: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 8: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 9: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 10: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 11: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 12: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 13: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 14: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false], 15: [version: "${base_image_version}.1", cpack_gen: "TGZ", source_package: false, other_var: false] ] @Field def toolchain_list = toolchain.keySet() as ArrayList @Field def toolchain_list_string = toolchain_list.join(',') pipeline { agent { label 'toolchain_docker' } parameters { extendedChoice( name: 'PLATFORM_LIST', description: 'Platforms to build for', type: 'PT_CHECKBOX', value: toolchain_list_string, defaultValue: toolchain_list_string, visibleItemCount: 15 ) } stages { stage('Build') { matrix { agent { dockerfile { dir 'jenkins/docker' additionalBuildArgs "--build-arg BASE_IMAGE=toolchain-${PLATFORM}:${toolchain["${PLATFORM}"].version} --build-arg USER_ID=\$(id -u) --build-arg GROUP_ID=\$(id -g)" reuseNode true args docker_arguments(toolchain["${PLATFORM}"].source_package) } } axes { axis { name 'PLATFORM' values 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'aa', 'ab', 'ac', 'ad', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15' } } when { beforeAgent true expression { def platform_list = params.PLATFORM_LIST.split(',') return platform_list.contains("${PLATFORM}") } } stages { stage('Build project') { steps { sh label: 'Configure', script: 'a' + "${PLATFORM} ${PLATFORM}" sh label: 'Build', script: "${PLATFORM}" } } } } } } }
I would say a big part of the problem is that you only get that "Method too large" message. You don't know if it's just over 64 KiB or if it's 1 MiB, you don't know what parts of your pipeline are contributing to the size.
A few things I found more or less surprising when simplifying this:
- The code inside the docker_arguments() function does matter. A suggestion when people find this issue is to put stuff into functions, so I thought that since it's an independent method it would not matter what's inside... but it did. Maybe it's because it's being called from inside agent/dockerfile?
- "${PLATFORM} ${PLATFORM}" vs "${PLATFORM}" in a step made a difference
- 'a' + 'b' vs 'ab' made a difference
- The number of elements in the PLATFORM axis makes a difference. I thought it would just be a "constant" called N times in parallel.
In the real pipeline, there are 30 platforms, not 45. But we will sooner or later reach 45, and in any case we already have problems with 30 since the real pipeline is not just this matrix.
All this is with SCRIPT_SPLITTING_TRANSFORMATION enabled. Which made things like
def docker_arguments(boolean with_source_package) { if(with_source_package) { return "--network none -v ${env.WORKSPACE}/src:${env.WORKSPACE}/src:ro,z" } else { return "--network none -v ${env.WORKSPACE}/src:${env.WORKSPACE}/src:ro,z" } } pipeline { agent { label 'toolchain_docker' } stages { stage('Test') { matrix { agent { dockerfile { dir 'router_agent/jenkins/docker' additionalBuildArgs "--build-arg BASE_IMAGE=docker.samknows.com/toolchain-base:11 --build-arg USER_ID=\$(id -u) --build-arg GROUP_ID=\$(id -g)" reuseNode true args docker_arguments(false) } } axes { axis { name 'PLATFORM' values '1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25','26','27','28','29','30' } axis { name 'BROWSER' values '1','2','3','4','5','6','7','8' } } when { beforeAgent true expression { return true } } stages { stage('Stage') { steps { echo "${PLATFORM} - ${BROWSER}" } } } } } } }
work, when before they didn't.
It doesn't pass params into pipeline with turned on flag when pipeline is inside shared library. Is there any solution for this case?