Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42829

[Regression] Jenkins pipeline start failing in on when/expression clause with custom method call with No such DSL method found among steps

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Major Major
    • None
    • Jenkins 2.50 (was updated from 2.49)
      Pipeline: Model Definition v 1.1.1 (was updated from 1.0.2)
      OS X 10.12

      Related stack overflow thread http://stackoverflow.com/questions/42822209/jenkins-pipeline-start-failing-in-on-when-expression-clause-with-custom-method-c/42824443#42824443

      After upgrading jenkins and it's plugins to the recent versions all my pipeline based jobs have start failing in the when/expression clause which were trying to call custom method defined in the current Jenkinsfile

       

      #!/usr/bin/env groovy
      
      pipeline {
      agent {
      label "mac"
      }
      parameters {
      booleanParam (
      name : "CONDITION",
      defaultValue: false,
      description: "")
      }
      
      stages {
      stage("Step 1") {
      when {
      expression {
      return condition()
      }
      }
      steps {
      externalStep()
      }
      }
      }
      }
      
      void externalStep() {
      echo "externalStep"
      }
      
      Boolean condition()
      {
      return params.CONDITION
      }
      

       

      My title

      {{java.lang.NoSuchMethodError: No such DSL method 'condition' found among steps [ansiColor, archive, bat, build, catchError, checkout, deleteDir, dir, dockerFingerprintFrom, dockerFingerprintRun, echo, emailext, emailextrecipients, envVarsForTool, error, fileExists, getContext, git, httpRequest, input, isUnix, library, libraryResource, load, mail, milestone, node, parallel, properties, publishHTML, pwd, readFile, readTrusted, resolveScm, retry, script, sh, slackSend, sleep, sshagent, stage, stash, step, svn, timeout, timestamps, tool, unarchive, unstash, validateDeclarativePipeline, waitUntil, withContext, withCredentials, withDockerContainer, withDockerRegistry, withDockerServer, withEnv, wrap, writeFile, ws] or symbols [all, allOf, always, androidLint, ant, antFromApache, antOutcome, antTarget, any, anyOf, apiToken, architecture, archiveArtifacts, artifactManager, batchFile, bitbucket, booleanParam, branch, buildButton, buildDiscarder, caseInsensitive, caseSensitive, choice, choiceParam, clock, cloud, command, configFile, configFileProvider, cron, crumb, defaultView, demand, disableConcurrentBuilds, docker, dockerfile, downloadSettings, downstream, dumb, envVars, environment, expression, file, fileParam, filePath, fingerprint, frameOptions, freeStyle, freeStyleJob, git, github, githubPush, gradle, hyperlink, hyperlinkToModels, installSource, jdk, jdkInstaller, jgit, jgitapache, jnlp, jobName, junit, label, lastDuration, lastFailure, lastGrantedAuthorities, lastStable, lastSuccess, legacy, legacySCM, list, local, location, logRotator, loggedInUsersCanDoAnything, masterBuild, maven, maven3Mojos, mavenErrors, mavenMojos, mavenWarnings, modernSCM, myView, node, nodeProperties, nonStoredPasswordParam, none, not, overrideIndexTriggers, paneStatus, parameters, password, pattern, pipeline-model, pipelineTriggers, plainText, plugin, pollSCM, projectNamingStrategy, proxy, queueItemAuthenticator, quietPeriod, run, runParam, schedule, scmRetryCount, search, security, shell, skipDefaultCheckout, skipStagesAfterUnstable, slave, stackTrace, standard, status, string, stringParam, swapSpace, text, textParam, tmpSpace, toolLocation, unsecured, upstream, usernameColonPassword, usernamePassword, viewsTabBar, weather, zfs, zip] or globals [currentBuild, docker, env, params, pipeline, scm] at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:149) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108) at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:21) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:115) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:103) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16) at Script1.run(Script1.groovy:1) at org.jenkinsci.plugins.pipeline.modeldefinition.when.impl.ExpressionConditionalScript.evaluate(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/when/impl/ExpressionConditionalScript.groovy:43) at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.evaluateWhen(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:420) at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.call(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:95) at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withEnvBlock(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:223) at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.withEnvBlock(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:222) at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.call(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy:94) at __cps.transform__(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:77) at sun.reflect.GeneratedMethodAccessor424.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:74) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:165) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:328) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Finished: FAILURE }}

          [JENKINS-42829] [Regression] Jenkins pipeline start failing in on when/expression clause with custom method call with No such DSL method found among steps

          Hi,

          Same kind of regression for me :

          [...]
          
          @NonCPS
          def initStages() {
              def jsonSlurper = new JsonSlurper()
              def parameters = jsonSlurper.parseText(params.STAGES)
          
              def stagesToRun = [
                      clean: parameters.fabrication.clean,
                      prepare: parameters.fabrication.prepare,
                      build: parameters.fabrication.build,
                      artifacts: parameters.fabrication.artifacts,
                      qa: parameters.tests.run
              ]
          
              return stagesToRun
          }
          
          def stagesToRun = initStages()
          
          [...]
          
          pipeline {
              agent {
                  label 'linux'
              }
              stages {
                  stage('Clean') {
                      when {
                          expression {
                              stagesToRun['clean']
                          }
                      }
          
          [...]

          Now it returns the error :

          groovy.lang.MissingPropertyException: No such property: stagesToRun for class: Script1
          	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
          	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:458)
          	at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:243)
          	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onGetProperty(GroovyInterceptor.java:52)
          	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:308)
          	at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241)
          	at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238)
          	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:28)
          	at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
          	at Script1.run(Script1.groovy:1)

          It was working like a charm before the last update to Jenkins 2.50 and Pipeline Model Definition to 1.1.1

           

          Florian Mignotet added a comment - Hi, Same kind of regression for me : [...] @NonCPS def initStages() { def jsonSlurper = new JsonSlurper() def parameters = jsonSlurper.parseText(params.STAGES) def stagesToRun = [ clean: parameters.fabrication.clean, prepare: parameters.fabrication.prepare, build: parameters.fabrication.build, artifacts: parameters.fabrication.artifacts, qa: parameters.tests.run ] return stagesToRun } def stagesToRun = initStages() [...] pipeline { agent { label 'linux' } stages { stage( 'Clean' ) { when { expression { stagesToRun[ 'clean' ] } } [...] Now it returns the error : groovy.lang.MissingPropertyException: No such property: stagesToRun for class: Script1 at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:458) at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:243) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onGetProperty(GroovyInterceptor.java:52) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:308) at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241) at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:28) at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20) at Script1.run(Script1.groovy:1) It was working like a charm before the last update to Jenkins 2.50 and Pipeline Model Definition to 1.1.1  

          Andrew Bayer added a comment -

          Yeah, this is an unpleasant side effect of the fix for https://issues.jenkins-ci.org/browse/JENKINS-42498 - I'll be working on this today, as well as adding more regression tests. Sorry for the inconvenience. =(

          Andrew Bayer added a comment - Yeah, this is an unpleasant side effect of the fix for https://issues.jenkins-ci.org/browse/JENKINS-42498  - I'll be working on this today, as well as adding more regression tests. Sorry for the inconvenience. =(

          Andrew Bayer added a comment -

          Andrew Bayer added a comment - I've added a fix for this to the PR for JENKINS-42753 at https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/140

          Andrew Bayer added a comment -

          So I'm not 100% sure that JENKINS-42753 is going to happen as such - but I've got a workaround for you anyway. Put your methods in a shared library, and then use the shared library via the libraries directive - i.e., for fmignotet's example, first, install the pipeline-utility-steps plugin to get the readJSON step, and then...

          vars/checkStagesToRun.groovy
          // shared library - vars/checkStagesToRun.groovy in the shared library repo
          def call(String stagesParam, String stageToCheck) {
              def parameters = readJSON(stagesParam)
          
              def stagesToRun = [
                      clean: parameters.fabrication.clean,
                      prepare: parameters.fabrication.prepare,
                      build: parameters.fabrication.build,
                      artifacts: parameters.fabrication.artifacts,
                      qa: parameters.tests.run
              ]
          
              return stagesToRun[stageToCheck]
          }
          

          Next, configure your shared library in Jenkins (see jenkins.io's docs for more info). Let's assume we're calling the library stageChecks and we're going to use the master branch.

          And then, in your Jenkinsfile:

          Jenkinsfile
          pipeline {
              agent {
                  label 'linux'
              }
              
              libraries {
                lib('stageChecks@master')
              }
              stages {
                  stage('Clean') {
                      when {
                          expression {
                              checkStagesToRun(params.STAGES, 'clean')
                          }
                      }
          
          [...]
          

          That will work. I just tested it. =) It could probably be simplified a bit here and there, but I wanted to get this general workaround to you ASAP.

          Andrew Bayer added a comment - So I'm not 100% sure that JENKINS-42753 is going to happen as such - but I've got a workaround for you anyway. Put your methods in a shared library, and then use the shared library via the libraries directive - i.e., for fmignotet 's example, first, install the pipeline-utility-steps plugin to get the readJSON step, and then... vars/checkStagesToRun.groovy // shared library - vars/checkStagesToRun.groovy in the shared library repo def call( String stagesParam, String stageToCheck) { def parameters = readJSON(stagesParam) def stagesToRun = [ clean: parameters.fabrication.clean, prepare: parameters.fabrication.prepare, build: parameters.fabrication.build, artifacts: parameters.fabrication.artifacts, qa: parameters.tests.run ] return stagesToRun[stageToCheck] } Next, configure your shared library in Jenkins (see jenkins.io's docs for more info). Let's assume we're calling the library stageChecks and we're going to use the master branch. And then, in your Jenkinsfile: Jenkinsfile pipeline { agent { label 'linux' } libraries { lib( 'stageChecks@master' ) } stages { stage( 'Clean' ) { when { expression { checkStagesToRun(params.STAGES, 'clean' ) } } [...] That will work. I just tested it. =) It could probably be simplified a bit here and there, but I wanted to get this general workaround to you ASAP.

          Great. I'll try it. Thanks you very much for the workaround.

          Florian Mignotet added a comment - Great. I'll try it. Thanks you very much for the workaround.

          Thank you abayer, I've found another solution inspired from your workaround. For me, this ticket can be closed.

          Florian Mignotet added a comment - Thank you abayer , I've found another solution inspired from your workaround. For me, this ticket can be closed.

          Liam Newman added a comment -

          fmignotet 

          What was your alternate work around? 

          Liam Newman added a comment - fmignotet   What was your alternate work around? 

          bitwiseman

          By replaying the pipeline (with the replay button), I've discovered that the code inside the "expression" block is now considered as another script.

          So, for my case, I've done like this :

           

          pipeline {
              agent {
                  label 'linux'
              }
              stages {
                  stage('Clean') {
                      when {
                          expression {
                              def cleanParameters = readJSON text: params.STAGES
                              return cleanParameters.fabrication.clean
                          }
                      }
          
          [...]
          

          Why "cleanParameters" variable name ?

           

          Because in each stage, I have a "when expression". If the same variable name is used with a "def myvar", the compilator doesn't like it. So a unique variable name in each "expression" block is necessary.

          It's a little bit dirty I know. But if I have to check an expression on steroids, I'll write it in a shared library

           

           

          Florian Mignotet added a comment - bitwiseman By replaying the pipeline (with the replay button), I've discovered that the code inside the "expression" block is now considered as another script. So, for my case, I've done like this :   pipeline { agent { label 'linux' } stages { stage( 'Clean' ) { when { expression { def cleanParameters = readJSON text: params.STAGES return cleanParameters.fabrication.clean } } [...] Why "cleanParameters" variable name ?   Because in each stage, I have a "when expression". If the same variable name is used with a "def myvar", the compilator doesn't like it. So a unique variable name in each "expression" block is necessary. It's a little bit dirty I know. But if I have to check an expression on steroids, I'll write it in a shared library    

          Anton Matosov added a comment -

          The shared library technology itself looks very overcomplicated to someone not familiar with java/groovy packaging (like me). Lack of documentation and no way to keep it side by side with Jenkinsfile makes it even less attractive.

          Introducing a shared library that should be managed outside of the main codebase just to have a couple helpers for when condition is doesn't seem to be feasible solution.  

          I have ended up wrapping all steps with script{} block and simply using "scriptable pipelines approach" with raw if/else conditions where I can call helpers without any problems. 

           

          Anton Matosov added a comment - The shared library technology itself looks very overcomplicated to someone not familiar with java/groovy packaging (like me). Lack of documentation and no way to keep it side by side with Jenkinsfile makes it even less attractive. Introducing a shared library that should be managed outside of the main codebase just to have a couple helpers for when condition is doesn't seem to be feasible solution.   I have ended up wrapping all steps with script{} block and simply using "scriptable pipelines approach" with raw if/else conditions where I can call helpers without any problems.   

          Liam Newman added a comment -

          fmignotet
          Thanks for the information! That's interesting that you need a unique variable name in each script block. I'll have to go take a look at that.

          antonmatosov
          Your point is well taken and has been raised before. It is unfortunate that you feel the need to resort to "script" directives.
          Please take a look at JENKINS-41335, JENKINS-42079, and JENKINS-41396 among others. Vote for those issues. Feel free to comment on them. User feedback and suggestions are vital to the improvement and direction of the project.

          Liam Newman added a comment - fmignotet Thanks for the information! That's interesting that you need a unique variable name in each script block. I'll have to go take a look at that. antonmatosov Your point is well taken and has been raised before. It is unfortunate that you feel the need to resort to "script" directives. Please take a look at JENKINS-41335 , JENKINS-42079 , and JENKINS-41396 among others. Vote for those issues. Feel free to comment on them. User feedback and suggestions are vital to the improvement and direction of the project.

          Andrew Bayer added a comment -

          FYI, closed this as WON'T FIX because Declarative doesn't officially support calling methods defined elsewhere in the Jenkinsfile. It's possible that may work in the future, but not guaranteed.

          Andrew Bayer added a comment - FYI, closed this as WON'T FIX because Declarative doesn't officially support calling methods defined elsewhere in the Jenkinsfile. It's possible that may work in the future, but not guaranteed.

          Dan Dean added a comment - - edited

          Just wanted to comment that not allowing function calls within `expression` blocks is severely limiting. My choices are: 

          • Duplicating complex expressions in a bunch of stages
          • Create an external shared library (huge learning curve and maintenance issue)
          • Convert the entire pipeline to Scripted instead of Declarative, significantly increasing the barrier to entry for the rest of my team

          These are all bad options. Allowing functional calls within `expression` blocks would make declarative pipelines much more powerful.

          Dan Dean added a comment - - edited Just wanted to comment that not allowing function calls within `expression` blocks is severely limiting. My choices are:  Duplicating complex expressions in a bunch of stages Create an external shared library (huge learning curve and maintenance issue) Convert the entire pipeline to Scripted instead of Declarative, significantly increasing the barrier to entry for the rest of my team These are all bad options. Allowing functional calls within `expression` blocks would make declarative pipelines much more powerful.

          Andrew Bayer added a comment -

          Fwiw, this will work in Declarative 1.2, and is already in place in the 1.2-beta-4 release.

          Andrew Bayer added a comment - Fwiw, this will work in Declarative 1.2, and is already in place in the 1.2-beta-4 release.

          Dan Dean added a comment -

          Thanks for the heads up abayer. Also, I neglected to mention that I absolutely love what switching to Pipelines and Blue Ocean has done for our CI processes.

          Dan Dean added a comment - Thanks for the heads up  abayer . Also, I neglected to mention that I absolutely love what switching to Pipelines and Blue Ocean has done for our CI processes.

          Liam Newman added a comment -

          Bulk closing resolved issues.

          Liam Newman added a comment - Bulk closing resolved issues.

            abayer Andrew Bayer
            antonmatosov Anton Matosov
            Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: