Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61389

Pipeline Matrix return a "Method code too large!" on a really short pipeline

    XMLWordPrintable

Details

    Description

      The following pipeline causes a "Method code too large!" error when you try to execute it, seems related to the environment variables and the number of axis but I dunno why.

      11:51:01  org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
      11:51:01  General error during class generation: Method code too large!
      11:51:01  
      11:51:01  java.lang.RuntimeException: Method code too large!
      11:51:01  	at groovyjarjarasm.asm.MethodWriter.a(Unknown Source)
      11:51:01  	at groovyjarjarasm.asm.ClassWriter.toByteArray(Unknown Source)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit$17.call(CompilationUnit.java:827)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      11:51:01  	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      11:51:01  	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      11:51:01  	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      11:51:01  	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:561)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:522)
      11:51:01  	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:327)
      11:51:01  	at hudson.model.ResourceController.execute(ResourceController.java:97)
      11:51:01  	at hudson.model.Executor.run(Executor.java:428)
      11:51:01  
      11:51:01  1 error
      11:51:01  
      11:51:01  	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      11:51:01  	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      11:51:01  	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      11:51:01  	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      11:51:01  	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      11:51:01  	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:561)
      11:51:01  	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:522)
      11:51:01  	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:327)
      11:51:01  	at hudson.model.ResourceController.execute(ResourceController.java:97)
      11:51:01  	at hudson.model.Executor.run(Executor.java:428)
      11:51:01  Finished: FAILURE
      
      #!/usr/bin/env groovy
      
      pipeline {
        agent { label 'linux' }
        stages {
          stage('Tests'){
            matrix {
              agent { label 'linux' }
              environment {
                TMPDIR = "/tmp"
                REUSE_CONTAINERS = "true"
                HOME = "/tmp"
                CONFIG_HOME = "/tmp"
                EC_WS ="/tmp/aaaa"
                VENV = "/tmp/.venv"
                PATH = "/tmp/aaaa/.ci/scripts:/tmp/bin:aaaa/bin:aaaa/.ci/scripts:${env.PATH}"
                CLUSTER_CONFIG_FILE="aaaaa/tests/environments/elastic_cloud.yml"
                ENABLE_ES_DUMP = "true"
              }
              axes {
                axis {
                    name 'TEST'
                    values 'all', 'dotnet', 'go', 'java', 'nodejs', 'python', 'ruby', 'rum'
                }
                axis {
                    name 'ELASTIC_STACK_VERSION'
                    values '8.0.0-SNAPSHOT', '7.7.0-SNAPSHOT', '7.6.1-SNAPSHOT', '6.8.7-SNAPSHOT'
                }
              }
              stages {
                stage('Prepare Test'){
                  steps {
                    echo "Running tests - ${ELASTIC_STACK_VERSION} x ${TEST}"
                  }
                }
              }
            }
          }
        }
      }
      
      

      Attachments

        Issue Links

          Activity

            ifernandezcalvo Ivan Fernandez Calvo added a comment - - edited

            Another case that does not make any sense that it said that the method is too large, the following pipeline return the "Method code too large!" error

            pipeline {
              agent { label 'ubuntu && immutable' }
              environment {
                BASE_DIR = 'src/github.com/elastic/beats'
                JOB_GCS_BUCKET = 'bucket'
                JOB_GCS_CREDENTIALS = 'creds'
                DOCKERELASTIC_SECRET = 'secret'
                DOCKER_REGISTRY = 'docker.example.com'
                SNAPSHOT = "true"
              }
              options {
                timeout(time: 3, unit: 'HOURS')
                buildDiscarder(logRotator(numToKeepStr: '20', artifactNumToKeepStr: '20', daysToKeepStr: '30'))
                timestamps()
                ansiColor('xterm')
                disableResume()
                durabilityHint('PERFORMANCE_OPTIMIZED')
                disableConcurrentBuilds()
              }
              triggers {
                issueCommentTrigger('(?i)^\\/packaging$')
              }
              stages {
                stage('Build Packages'){
                  matrix {
                    agent { label 'ubuntu && immutable' }
                    axes {
                      axis {
                        name 'PLATFORMS'
                        values (
                          '+linux/armv7',
                          '+linux/ppc64le',
                          '+linux/s390x',
                          '+linux/mips64',
                          '+darwin',
                          '+windows/386',
                          '+windows/amd64'
                        )
                      }
                      axis {
                        name 'BEATS_FOLDER'
                        values (
                          'auditbeat',
                          'filebeat',
                          'heartbeat',
                          'journalbeat',
                          'metricbeat',
                          'packetbeat',
                          'winlogbeat',
                          'x-pack/auditbeat',
                          'x-pack/filebeat',
                          'x-pack/functionbeat',
                          'x-pack/heartbeat',
                          'x-pack/journalbeat',
                          'x-pack/metricbeat',
                          'x-pack/packetbeat',
                          'x-pack/winlogbeat'
                        )
                      }
                    }
                    stages {
                      stage('Package'){
                        options { skipDefaultCheckout() }
                        environment {
                          HOME = "${env.WORKSPACE}"
                        }
                        steps {
                          echo "Running tests"
                        }
                      }
                    }
                  }
                }
              }
            }
            

            moving the agent request from the matrix to the stage resolves the issue, but it should be supported

            pipeline {
              agent { label 'ubuntu && immutable' }
              environment {
                BASE_DIR = 'src/github.com/elastic/beats'
                JOB_GCS_BUCKET = 'bucket'
                JOB_GCS_CREDENTIALS = 'creds'
                DOCKERELASTIC_SECRET = 'secret'
                DOCKER_REGISTRY = 'docker.example.com'
                SNAPSHOT = "true"
              }
              options {
                timeout(time: 3, unit: 'HOURS')
                buildDiscarder(logRotator(numToKeepStr: '20', artifactNumToKeepStr: '20', daysToKeepStr: '30'))
                timestamps()
                ansiColor('xterm')
                disableResume()
                durabilityHint('PERFORMANCE_OPTIMIZED')
                disableConcurrentBuilds()
              }
              triggers {
                issueCommentTrigger('(?i)^\\/packaging$')
              }
              stages {
                stage('Build Packages'){
                  matrix {
                    axes {
                      axis {
                        name 'PLATFORMS'
                        values (
                          '+linux/armv7',
                          '+linux/ppc64le',
                          '+linux/s390x',
                          '+linux/mips64',
                          '+darwin',
                          '+windows/386',
                          '+windows/amd64'
                        )
                      }
                      axis {
                        name 'BEATS_FOLDER'
                        values (
                          'auditbeat',
                          'filebeat',
                          'heartbeat',
                          'journalbeat',
                          'metricbeat',
                          'packetbeat',
                          'winlogbeat',
                          'x-pack/auditbeat',
                          'x-pack/filebeat',
                          'x-pack/functionbeat',
                          'x-pack/heartbeat',
                          'x-pack/journalbeat',
                          'x-pack/metricbeat',
                          'x-pack/packetbeat',
                          'x-pack/winlogbeat'
                        )
                      }
                    }
                    stages {
                      stage('Package'){
                        agent { label 'ubuntu && immutable' }
                        options { skipDefaultCheckout() }
                        environment {
                          HOME = "${env.WORKSPACE}"
                        }
                        steps {
                          echo "Running tests"
                        }
                      }
                    }
                  }
                }
              }
            }
            
            ifernandezcalvo Ivan Fernandez Calvo added a comment - - edited Another case that does not make any sense that it said that the method is too large, the following pipeline return the "Method code too large!" error pipeline { agent { label 'ubuntu && immutable' } environment { BASE_DIR = 'src/github.com/elastic/beats' JOB_GCS_BUCKET = 'bucket' JOB_GCS_CREDENTIALS = 'creds' DOCKERELASTIC_SECRET = 'secret' DOCKER_REGISTRY = 'docker.example.com' SNAPSHOT = " true " } options { timeout(time: 3, unit: 'HOURS' ) buildDiscarder(logRotator(numToKeepStr: '20' , artifactNumToKeepStr: '20' , daysToKeepStr: '30' )) timestamps() ansiColor( 'xterm' ) disableResume() durabilityHint( 'PERFORMANCE_OPTIMIZED' ) disableConcurrentBuilds() } triggers { issueCommentTrigger( '(?i)^\\/packaging$' ) } stages { stage( 'Build Packages' ){ matrix { agent { label 'ubuntu && immutable' } axes { axis { name 'PLATFORMS' values ( '+linux/armv7' , '+linux/ppc64le' , '+linux/s390x' , '+linux/mips64' , '+darwin' , '+windows/386' , '+windows/amd64' ) } axis { name 'BEATS_FOLDER' values ( 'auditbeat' , 'filebeat' , 'heartbeat' , 'journalbeat' , 'metricbeat' , 'packetbeat' , 'winlogbeat' , 'x-pack/auditbeat' , 'x-pack/filebeat' , 'x-pack/functionbeat' , 'x-pack/heartbeat' , 'x-pack/journalbeat' , 'x-pack/metricbeat' , 'x-pack/packetbeat' , 'x-pack/winlogbeat' ) } } stages { stage( 'Package' ){ options { skipDefaultCheckout() } environment { HOME = "${env.WORKSPACE}" } steps { echo "Running tests" } } } } } } } moving the agent request from the matrix to the stage resolves the issue, but it should be supported pipeline { agent { label 'ubuntu && immutable' } environment { BASE_DIR = 'src/github.com/elastic/beats' JOB_GCS_BUCKET = 'bucket' JOB_GCS_CREDENTIALS = 'creds' DOCKERELASTIC_SECRET = 'secret' DOCKER_REGISTRY = 'docker.example.com' SNAPSHOT = " true " } options { timeout(time: 3, unit: 'HOURS' ) buildDiscarder(logRotator(numToKeepStr: '20' , artifactNumToKeepStr: '20' , daysToKeepStr: '30' )) timestamps() ansiColor( 'xterm' ) disableResume() durabilityHint( 'PERFORMANCE_OPTIMIZED' ) disableConcurrentBuilds() } triggers { issueCommentTrigger( '(?i)^\\/packaging$' ) } stages { stage( 'Build Packages' ){ matrix { axes { axis { name 'PLATFORMS' values ( '+linux/armv7' , '+linux/ppc64le' , '+linux/s390x' , '+linux/mips64' , '+darwin' , '+windows/386' , '+windows/amd64' ) } axis { name 'BEATS_FOLDER' values ( 'auditbeat' , 'filebeat' , 'heartbeat' , 'journalbeat' , 'metricbeat' , 'packetbeat' , 'winlogbeat' , 'x-pack/auditbeat' , 'x-pack/filebeat' , 'x-pack/functionbeat' , 'x-pack/heartbeat' , 'x-pack/journalbeat' , 'x-pack/metricbeat' , 'x-pack/packetbeat' , 'x-pack/winlogbeat' ) } } stages { stage( 'Package' ){ agent { label 'ubuntu && immutable' } options { skipDefaultCheckout() } environment { HOME = "${env.WORKSPACE}" } steps { echo "Running tests" } } } } } } }

            I'm running into this now as well

            henryborchers Henry Borchers added a comment - I'm running into this now as well

            It does seem to be related to having a large number of axis values, and I assume Groovy can't compile all of that at once. When I remove any one of the axis values from your script, it does run as expected.

            rensgroothuijsen Rens Groothuijsen added a comment - It does seem to be related to having a large number of axis values, and I assume Groovy can't compile all of that at once. When I remove any one of the axis values from your script, it does run as expected.
            bitwiseman Liam Newman added a comment -

            rensgroothuijsen
            ifernandezcalvo
            This is not due to the size of Jenkinsfile but a limit in the JVM on the size of the compiled pipeline. There is a workaround available in the Declarative that can be turned on using the feature flag:
            org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true

            Please see JENKINS-37984 . I'm resolving this as a duplicate of that issue.

            bitwiseman Liam Newman added a comment - rensgroothuijsen ifernandezcalvo This is not due to the size of Jenkinsfile but a limit in the JVM on the size of the compiled pipeline. There is a workaround available in the Declarative that can be turned on using the feature flag: org.jenkinsci.plugins.pipeline.modeldefinition.parser.RuntimeASTTransformer.SCRIPT_SPLITTING_TRANSFORMATION=true Please see JENKINS-37984 . I'm resolving this as a duplicate of that issue.

            this workaround has a side effect to take in count before enabling this flag.

            WARNING: this is a global change. If you install this version an have SCRIPT_SPLITTING_TRANSFORMATION=true, any declarative pipeline Jenkinsfile with a local variable declaration outside the pipeline block will fail. You may switch back to allowing locally declared variables by setting SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES=true.
            
            ifernandezcalvo Ivan Fernandez Calvo added a comment - this workaround has a side effect to take in count before enabling this flag. WARNING: this is a global change. If you install this version an have SCRIPT_SPLITTING_TRANSFORMATION= true , any declarative pipeline Jenkinsfile with a local variable declaration outside the pipeline block will fail. You may switch back to allowing locally declared variables by setting SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES= true .
            bitwiseman Liam Newman added a comment -

            ifernandezcalvo
            Yes. This is meant to give users clearer feedback when using script splitting - there have a been a number of people who were unaware they were doing something that was defeating script splitting. Also, the fix for the error is to add @Field to any def variableName-style variables. Very easy and also a best practice.
            Finally, if you set SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES=true you can silence the error.

            Have you tried the change? I have heard very little feedback.

            bitwiseman Liam Newman added a comment - ifernandezcalvo Yes. This is meant to give users clearer feedback when using script splitting - there have a been a number of people who were unaware they were doing something that was defeating script splitting. Also, the fix for the error is to add @Field to any def variableName -style variables. Very easy and also a best practice. Finally, if you set SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES=true you can silence the error. Have you tried the change? I have heard very little feedback.
            ifernandezcalvo Ivan Fernandez Calvo added a comment - - edited

            no, we workaround it moving the logic to functions or steps in the pipeline shared library, for the most complex pipeline we have (a big monorepo) we ended making an opinionated YAML file we interpreted from a Jenkinsfile, this gives us no limit of code.

            https://github.com/elastic/beats/pull/20104

            ifernandezcalvo Ivan Fernandez Calvo added a comment - - edited no, we workaround it moving the logic to functions or steps in the pipeline shared library, for the most complex pipeline we have (a big monorepo) we ended making an opinionated YAML file we interpreted from a Jenkinsfile, this gives us no limit of code. https://github.com/elastic/beats/pull/20104
            reddwarf94 Cristian added a comment - - edited

            FWIW even with SCRIPT_SPLITTING_TRANSFORMATION (and without SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES) I'm getting the error with a matrix of a single axis with 30 values.

            I would have expected the code size to basically stay constant when increasing the number of values. But obviously I was wrong.

            In fact, reducing the number of axis values solves the issue... but removing parts of the pipeline outside the matrix also solve the issue. It's all a bit random.

            What can be done to better understand how the pipeline is transformed into these 64KiB+ classes? Blindly starting to move code into functions is not ideal.

            reddwarf94 Cristian added a comment - - edited FWIW even with SCRIPT_SPLITTING_TRANSFORMATION (and without SCRIPT_SPLITTING_ALLOW_LOCAL_VARIABLES) I'm getting the error with a matrix of a single axis with 30 values. I would have expected the code size to basically stay constant when increasing the number of values. But obviously I was wrong. In fact, reducing the number of axis values solves the issue... but removing parts of the pipeline outside the matrix also solve the issue. It's all a bit random. What can be done to better understand how the pipeline is transformed into these 64KiB+ classes? Blindly starting to move code into functions is not ideal.

            People

              Unassigned Unassigned
              ifernandezcalvo Ivan Fernandez Calvo
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: