Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32766

groovy.xml.MarkupBuilder fails in Pipeline with NoSuchMethodError

      http://www.groovy-lang.org/processing-xml.html

      def writer = new java.io.StringWriter()
      def xml = new groovy.xml.MarkupBuilder(writer) 
      
      xml.records() { 
          car(name:'HSV Maloo', make:'Holden', year:2006) {
              country('Australia')
              record(type:'speed', 'Production Pickup Truck with speed of 271kph')
          }
          car(name:'Royale', make:'Bugatti', year:1931) {
              country('France')
              record(type:'price', 'Most Valuable Car at $15 million')
          }
      }
      
      def records = new XmlSlurper().parseText(writer.toString()) 
      
      println "==="
      println writer
      
      java.lang.NoSuchMethodError: No such DSL method 'car' found among [archive, bat, build, catchError, checkout, checkpoint, deleteDir, dir, dockerFingerprintFrom, dockerFingerprintRun, echo, error, fileExists, git, input, isUnix, load, mail, node, parallel, properties, pwd, readFile, retry, sh, sleep, sshagent, stage, stash, step, timeout, tool, triggerRemoteJob, unarchive, unstash, waitUntil, withCredentials, withDockerContainer, withDockerRegistry, withDockerServer, withEnv, wrap, writeFile, ws]
      	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:107)
      	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:112)
      	at groovy.lang.GroovyObject$invokeMethod$0.call(Unknown Source)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
      	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151)
      	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:21)
      	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:76)
      	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
      	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:15)
      	at WorkflowScript.run(WorkflowScript:5)
      	at WorkflowScript.run(WorkflowScript:4)
      	at ___cps.transform___(Native Method)
      	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:55)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
      	at sun.reflect.GeneratedMethodAccessor182.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      	at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:40)
      	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
      	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:19)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
      	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:106)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
      	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:277)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:77)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:186)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:184)
      	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
      	...
      Finished: FAILURE
      

          [JENKINS-32766] groovy.xml.MarkupBuilder fails in Pipeline with NoSuchMethodError

          I have following result when writing file using delegate:

          Code:

          import groovy.xml.*
          
          node("base") {
              wrap([$class: 'TimestamperBuildWrapper']) {
                  stage("XML") {
                      print createOverwrite()
                      writeFile(file: 'project.xml', text: createOverwrite())
                      archiveArtifacts(artifacts: 'project.xml', excludes: null)
                  }
              }
          }
          
          def createOverwrite() {
              def stringWriter = new StringWriter()
              def manifestBuilder = new MarkupBuilder(stringWriter)
          
              manifestBuilder.mkp.xmlDeclaration(version: "1.0", encoding: "utf-8")
              manifestBuilder.manifest {
                  delegate.'remove-project'(name: 'project/project') {
                  }
                  delegate.'project'(path: 'project', name: 'project/project', remote: 'gitserver', revision: 'refs/heads/master') {
                  }
              }
          
              return stringWriter.toString()
          }
          

          Output file:

          <?xml version='1.0' encoding='utf-8'?>
          <manifest>
          <remove-project name='project/project'>
              <project path='project' name='project/project' remote='gitlab' revision='refs/heads/master'
          

          Expected output:

          <?xml version='1.0' encoding='utf-8'?>
          <manifest>
            <remove-project name='project/project' />
            <project path='project' name='project/project' remote='gitserver' revision='refs/heads/master' />
          </manifest>
          

           

          Grzegorz Zieba added a comment - I have following result when writing file using delegate: Code: import groovy.xml.* node( "base" ) { wrap([$class: 'TimestamperBuildWrapper' ]) { stage( "XML" ) { print createOverwrite() writeFile(file: 'project.xml' , text: createOverwrite()) archiveArtifacts(artifacts: 'project.xml' , excludes: null ) } } } def createOverwrite() { def stringWriter = new StringWriter() def manifestBuilder = new MarkupBuilder(stringWriter) manifestBuilder.mkp.xmlDeclaration(version: "1.0" , encoding: "utf-8" ) manifestBuilder.manifest { delegate. 'remove-project' (name: 'project/project' ) { } delegate. 'project' (path: 'project' , name: 'project/project' , remote: 'gitserver' , revision: 'refs/heads/master' ) { } } return stringWriter.toString() } Output file: <?xml version='1.0' encoding='utf-8'?> <manifest> <remove-project name='project/project'> <project path='project' name='project/project' remote='gitlab' revision='refs/heads/master' Expected output: <?xml version= '1.0' encoding= 'utf-8' ?> <manifest> <remove-project name= 'project/project' /> <project path= 'project' name= 'project/project' remote= 'gitserver' revision= 'refs/heads/master' /> </manifest>  

          grzzie Try adding @NonCPS annotation to createOverwrite method, it is working.

          And also I have noticed another interesting thing, delegate doesn't work when we have a logic which needs to have a loops(I guess thats the problem.) The hack is add multiple delegates to the one those are in internal loops.

          import groovy.xml.*
          import static java.util.UUID.randomUUID 
          
          node() {
              stage("GenerateXML") {
                  def command = 'Testing'
                  def params = [name: 'Testing',
                                 build: 'TestBuild',
                                 actualParameter: [
                                          [actualParameterName: 'Param1', value: 'Param1Value'],
                                          [actualParameterName: 'Param2', value: 'Param2Value']]]
                  echo createXML(command, params)
              }
          }
          
          @NonCPS
          def createXML(command, params) {
              def stringWriter = new StringWriter()
              def requestBuilder = new MarkupBuilder(stringWriter)
              requestBuilder.mkp.xmlDeclaration(version: "1.0", encoding: "utf-8")
              requestBuilder
                      .'requests'('xsi:noNamespaceSchemaLocation':'test.xsd', 'timeOut':'180', 'version':'2.0', 'xmlns:xsi':'http://www.w3.org/2001/XMLSchema-instance') {
                          delegate.'request'(requestId: randomUUID() as String) {
                              delegate."$command"() {
                                  params.each { k1, v1 ->
                                      if (v1 instanceof List) {
                                          v1.each { v2 ->
                                              delegate.delegate.delegate."$k1"() {
                                                  v2.each { k3, v3 ->
                                                      delegate.delegate."$k3"("$v3")
                                                  }
                                              }
                                          }
                                      } else {
                                          delegate.delegate."$k1"("$v1")
                                      }
                                  }
                              }
                          }
                      }
              return stringWriter.toString()
          }
          

          Naresh Rayapati added a comment - grzzie Try adding @NonCPS annotation to createOverwrite method, it is working. And also I have noticed another interesting thing, delegate doesn't work when we have a logic which needs to have a loops(I guess thats the problem.) The hack is add multiple delegates to the one those are in internal loops. import groovy.xml.* import static java.util.UUID.randomUUID node() { stage( "GenerateXML" ) { def command = 'Testing' def params = [name: 'Testing' , build: 'TestBuild' , actualParameter: [ [actualParameterName: 'Param1' , value: 'Param1Value' ], [actualParameterName: 'Param2' , value: 'Param2Value' ]]] echo createXML(command, params) } } @NonCPS def createXML(command, params) { def stringWriter = new StringWriter() def requestBuilder = new MarkupBuilder(stringWriter) requestBuilder.mkp.xmlDeclaration(version: "1.0" , encoding: "utf-8" ) requestBuilder . 'requests' ( 'xsi:noNamespaceSchemaLocation' : 'test.xsd' , 'timeOut' : '180' , 'version' : '2.0' , 'xmlns:xsi' : 'http: //www.w3.org/2001/XMLSchema-instance' ) { delegate. 'request' (requestId: randomUUID() as String ) { delegate. "$command" () { params.each { k1, v1 -> if (v1 instanceof List) { v1.each { v2 -> delegate.delegate.delegate. "$k1" () { v2.each { k3, v3 -> delegate.delegate. "$k3" ( "$v3" ) } } } } else { delegate.delegate. "$k1" ( "$v1" ) } } } } } return stringWriter.toString() }

          Andrew Bayer added a comment -

          Yeah, MarkupBuilder will never work right in a CPS context. It always needs to be invoked within a method annotated with @NonCPS. Sorry.

          Andrew Bayer added a comment - Yeah, MarkupBuilder will never work right in a CPS context. It always needs to be invoked within a method annotated with @NonCPS . Sorry.

          Francois Ferrand added a comment - - edited

          Working only inside @NonCPS methods is fine; but even this works only partially: it requires explicitely referencing delegate

          Francois Ferrand added a comment - - edited Working only inside @NonCPS methods is fine; but even this works only partially: it requires explicitely referencing delegate

          neiltingley added a comment -

          Same problem and the above worked for me. 

          neiltingley added a comment - Same problem and the above worked for me. 

          Same issue with html formatting, in script console ParseToHTMLTable method from below works fine:

           

          import groovy.xml.MarkupBuilder
          def ParseToHTMLTable(headers, params) {
            def writer = new StringWriter()
            new MarkupBuilder(writer).table() {
              tr {headers.each { title -> th(title) }}
              params.each { row ->
                tr {
                  td(row.key)
                  td(row.value)
                }
              }
            }
            return writer.toString()
          }
          
          def params = [
            "Job": "JobName",
            "isInQueue": true,
            "isBuilding": true,
            "isBuildBlocked": false,
            "Builds count": 15,
            "IsBuilding count": 1,
            "InProgress count": 1
          ]
          
          println ParseToHTMLTable(["Parameter","Value"], params)
          

          but in pipeline it requires multiple delegates:

           

           

          import groovy.xml.MarkupBuilder
          @NonCPS
          def ParseToHTMLTable(headers, params) {
            def writer = new StringWriter()
            new MarkupBuilder(writer).table() {
              delegate.tr {headers.each { title -> delegate.delegate.th(title) }}
              params.each { row ->
                delegate.delegate.tr {
                  delegate.td(row.key)
                  delegate.td(row.value)
                }
              }
            }
            return writer.toString()
          }
          

           

           

          Jakub Pawlinski added a comment - Same issue with html formatting, in script console ParseToHTMLTable method from below works fine:   import groovy.xml.MarkupBuilder def ParseToHTMLTable(headers, params) { def writer = new StringWriter() new MarkupBuilder(writer).table() {     tr {headers.each { title -> th(title) }}     params.each { row ->       tr {         td(row.key)         td(row.value)       }     }   }    return writer.toString() } def params = [    "Job" : "JobName" ,    "isInQueue" : true ,    "isBuilding" : true ,    "isBuildBlocked" : false ,    "Builds count" : 15,    "IsBuilding count" : 1,    "InProgress count" : 1 ] println ParseToHTMLTable([ "Parameter" , "Value" ], params) but in pipeline it requires multiple delegates:     import groovy.xml.MarkupBuilder @NonCPS def ParseToHTMLTable(headers, params) {   def writer = new StringWriter()    new MarkupBuilder(writer).table() {     delegate.tr {headers.each { title -> delegate.delegate.th(title) }}     params.each { row ->       delegate.delegate.tr {         delegate.td(row.key)         delegate.td(row.value)       }     }   }    return writer.toString() }    

          Hari Dara added a comment - - edited

          I want to try the delegate workaround, but getting this error: 

          org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new groovy.xml.MarkupBuilder java.io.Writer 

          It doesn't seem like this is whitelisted, so how is it working for others?

          Hari Dara added a comment - - edited I want to try the delegate workaround, but getting this error:  org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new groovy.xml.MarkupBuilder java.io.Writer It doesn't seem like this is whitelisted , so how is it working for others?

          haridsv you have to approve it yourself at http://yourhost:8080/scriptApproval/ or jenkins / manage jenkins / in-process script approval

          Jakub Pawlinski added a comment - haridsv you have to approve it yourself at http://yourhost:8080/scriptApproval/  or jenkins / manage jenkins / in-process script approval

          Hari Dara added a comment -

          Thanks quas, I am aware of that but was downplaying it as I don't really prefer that approach in a multi-tenant environment with no admin access. Won't it be safe to whitelist it?

          Hari Dara added a comment - Thanks quas , I am aware of that but was downplaying it as I don't really prefer that approach in a multi-tenant environment with no admin access. Won't it be safe to whitelist it?

          Andrew Garner added a comment -

          This workaround using multiple delegates seems to be broken again. It was working for me using LTS v2.150.1 but with more recent releases and updates plugins it is not working again. I don't know whether it is the core Jenkins version or a plugin version that has reverted this issue.

          I've changed my approach to use groovy.xml.DOMBuilder instead. The code is more verbose and it still requires many methods to be whitelisted, but I hope it is a more stable approach. See https://github.com/agarthetiger/jengu/blob/d9dddba3d1b7644216d088413638dbc84fba1ab4/vars/libraryTestRunner.groovy#L143

          Andrew Garner added a comment - This workaround using multiple delegates seems to be broken again. It was working for me using LTS v2.150.1 but with more recent releases and updates plugins it is not working again. I don't know whether it is the core Jenkins version or a plugin version that has reverted this issue. I've changed my approach to use groovy.xml.DOMBuilder instead. The code is more verbose and it still requires many methods to be whitelisted, but I hope it is a more stable approach. See https://github.com/agarthetiger/jengu/blob/d9dddba3d1b7644216d088413638dbc84fba1ab4/vars/libraryTestRunner.groovy#L143

            kohsuke Kohsuke Kawaguchi
            cleclerc Cyrille Le Clerc
            Votes:
            16 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated: