Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26481

Mishandling of binary methods accepting Closure

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      At least some closures are executed only once inside of Groovy CPS DSL scripts managed by the workflow plugin.

      Steps to reproduce:
      1. Create a new workflow with the following script:
      node {
      [1, 2, 3].each

      { println it }

      println "abc".replaceAll(/[a-z]/)

      { it.toUpperCase() }

      }
      2. Build the workflow

      Actual output:
      Started by user anonymous
      Running: Allocate node : Start
      Running on master in /var/lib/jenkins/jobs/testflow/workspace
      Running: Allocate node : Body : Start
      Running: Print Message
      1
      Running: Print Message
      A
      Running: Allocate node : Body : End
      Running: Allocate node : End
      Running: End of Workflow
      Finished: SUCCESS

      Expected output:
      Started by user anonymous
      Running: Allocate node : Start
      Running on master in /var/lib/jenkins/jobs/testflow/workspace
      Running: Allocate node : Body : Start
      Running: Print Message
      1
      Running: Print Message
      2
      Running: Print Message
      3
      Running: Print Message
      ABC
      Running: Allocate node : Body : End
      Running: Allocate node : End
      Running: End of Workflow
      Finished: SUCCESS

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            Sounds like a bad implementation of something in the groovy-cps library.

            Show
            jglick Jesse Glick added a comment - Sounds like a bad implementation of something in the groovy-cps library.
            Hide
            jglick Jesse Glick added a comment -

            JENKINS-26300 sounds similar, perhaps with the same root cause.

            Show
            jglick Jesse Glick added a comment - JENKINS-26300 sounds similar, perhaps with the same root cause.
            Hide
            jglick Jesse Glick added a comment -

            groovy-cps #8 covers the particular case reported here: there is an implementation of each for CPS-transformed code, but for reasons still TBD it is not getting used when called on an array (it is used for example on sequences).

            Show
            jglick Jesse Glick added a comment - groovy-cps #8 covers the particular case reported here: there is an implementation of each for CPS-transformed code, but for reasons still TBD it is not getting used when called on an array (it is used for example on sequences).
            Hide
            jglick Jesse Glick added a comment -

            groovy-cps #9 would allow closures to run synchronously, permitting routine functional-style programs to be used so long as the closures were not trying to do long-running tasks like calling shell steps.

            Show
            jglick Jesse Glick added a comment - groovy-cps #9 would allow closures to run synchronously, permitting routine functional-style programs to be used so long as the closures were not trying to do long-running tasks like calling shell steps.
            Hide
            jglick Jesse Glick added a comment -

            groovy-cps #7 would allow us to offer CPS-transformed versions of all standard Groovy closure-based methods like find without having to reimplement each one by hand.

            Show
            jglick Jesse Glick added a comment - groovy-cps #7 would allow us to offer CPS-transformed versions of all standard Groovy closure-based methods like find without having to reimplement each one by hand.
            Hide
            rawlingsj James Rawlings added a comment -

            I think I've just hit this same problem, using the workflow plugin I'm reading a multiline file but unable to loop over each line, I only get one line printed in this example..

            def file = readFile('MultiLineFile.txt')
            file.eachLine { line ->
              println line
            }
            
            Show
            rawlingsj James Rawlings added a comment - I think I've just hit this same problem, using the workflow plugin I'm reading a multiline file but unable to loop over each line, I only get one line printed in this example.. def file = readFile( 'MultiLineFile.txt' ) file.eachLine { line -> println line }
            Hide
            jglick Jesse Glick added a comment -

            That is the same. You need to split and use a C-style loop.

            Show
            jglick Jesse Glick added a comment - That is the same. You need to split and use a C-style loop.
            Hide
            rawlingsj James Rawlings added a comment -

            Yeah figured it was worth mentioning and thanks for the original comments as it helped me move on. For completeness and to complement above answer here's what I used..

            def lines = new String(multiLineFile).split( '\n' )
            for(int i = 0; i < lines.size(); i++){
              println lines[i]
            }
            
            Show
            rawlingsj James Rawlings added a comment - Yeah figured it was worth mentioning and thanks for the original comments as it helped me move on. For completeness and to complement above answer here's what I used.. def lines = new String (multiLineFile).split( '\n' ) for ( int i = 0; i < lines.size(); i++){ println lines[i] }
            Hide
            jglick Jesse Glick added a comment -

            That works, though note that new String(multiLineFile) is redundant; you can just use multiLineFile directly.

            Show
            jglick Jesse Glick added a comment - That works, though note that new String(multiLineFile) is redundant; you can just use multiLineFile directly.
            Hide
            rawlingsj James Rawlings added a comment -

            Awesome - thanks!

            Show
            rawlingsj James Rawlings added a comment - Awesome - thanks!
            Hide
            fchuong Frédéric Chuong added a comment -

            workflow-plugin 1.10, using @com.cloudbees.groovy.cps.NonCPS does produce the expected output (when combined with synchronous steps such as org.jenkinsci.plugins.workflow.steps.EchoStep.Execution) as a workaround to groovy-cps #9.

            @com.cloudbees.groovy.cps.NonCPS
            def foo1() {
              [1, 2, 3].each {
                println it
              }
            }
            
            @com.cloudbees.groovy.cps.NonCPS
            def foo2() {
              println "abc".replaceAll(/[a-z]/) {
                it.toUpperCase()
              }
            }
            
            node {
              foo1()
              foo2()
            }
            
            Started by user anonymous
            Running: Allocate node : Start
            Running on master in xxx
            Running: Allocate node : Body : Start
            Running: Print Message
            1
            Running: Print Message
            2
            Running: Print Message
            3
            Running: Print Message
            ABC
            Running: Allocate node : Body : End
            Running: Allocate node : End
            Running: End of Workflow
            Finished: SUCCESS
            
            Show
            fchuong Frédéric Chuong added a comment - workflow-plugin 1.10, using @com.cloudbees.groovy.cps.NonCPS does produce the expected output (when combined with synchronous steps such as org.jenkinsci.plugins.workflow.steps.EchoStep.Execution ) as a workaround to groovy-cps #9 . @com.cloudbees.groovy.cps.NonCPS def foo1() { [1, 2, 3].each { println it } } @com.cloudbees.groovy.cps.NonCPS def foo2() { println "abc" .replaceAll(/[a-z]/) { it.toUpperCase() } } node { foo1() foo2() } Started by user anonymous Running: Allocate node : Start Running on master in xxx Running: Allocate node : Body : Start Running: Print Message 1 Running: Print Message 2 Running: Print Message 3 Running: Print Message ABC Running: Allocate node : Body : End Running: Allocate node : End Running: End of Workflow Finished: SUCCESS
            Hide
            jglick Jesse Glick added a comment -

            Frédéric Chuong while it may work to run certain steps inside a @NonCPS method (generally only those using AbstractSynchronousStepExecution: echo, error, pwd), in general it will fail horribly, so my recommendation is to never call any step from inside such a method: just use it to do some computation and then pass the result back to the flow. Ideally the engine would automatically detect the attempt and reject it.

            Show
            jglick Jesse Glick added a comment - Frédéric Chuong while it may work to run certain steps inside a @NonCPS method (generally only those using AbstractSynchronousStepExecution : echo , error , pwd ), in general it will fail horribly, so my recommendation is to never call any step from inside such a method: just use it to do some computation and then pass the result back to the flow. Ideally the engine would automatically detect the attempt and reject it.
            Hide
            fchuong Frédéric Chuong added a comment -

            Yes, which is why synchronous steps were mentioned when using @NonCPS as the current implementation already fails with basic Groovy:

            @com.cloudbees.groovy.cps.NonCPS
            def doAssertNoCPS() {
              // no error when CPS disabled
              assert [1, 2, 3].collect { it } == [1, 2, 3]
            }
            def doAssertCPS() {
              // fails because CPS transformation results in '4'
              assert [4, 5, 6].collect { it } == [4, 5, 6]
            }
            
            doAssertNoCPS()
            doAssertCPS()
            

            To be more relevant with current issue, it prevents such direct Groovy approach:

            // won't retrieve parameter value as result of 'find' is inconsistent
            currentBuild.rawBuild.getAction(hudson.model.ParametersAction).parameters.find { it.name == 'FOO' }.value
            
            Show
            fchuong Frédéric Chuong added a comment - Yes, which is why synchronous steps were mentioned when using @NonCPS as the current implementation already fails with basic Groovy: @com.cloudbees.groovy.cps.NonCPS def doAssertNoCPS() { // no error when CPS disabled assert [1, 2, 3].collect { it } == [1, 2, 3] } def doAssertCPS() { // fails because CPS transformation results in '4' assert [4, 5, 6].collect { it } == [4, 5, 6] } doAssertNoCPS() doAssertCPS() To be more relevant with current issue, it prevents such direct Groovy approach: // won 't retrieve parameter value as result of ' find' is inconsistent currentBuild.rawBuild.getAction(hudson.model.ParametersAction).parameters.find { it.name == 'FOO' }.value
            Hide
            eholzwarth Ed Holzwarth added a comment -

            I'm not understanding the workaround, apparently. I tried adding the NonCPS annotation as follows:

            		@com.cloudbees.groovy.cps.NonCPS 
            		def targetsNotYetDeployed = config.deployTargets.findAll{ !it.hasProperty("deployed") }
            		def done = (targetsNotYetDeployed.size() == 0)
            

            But this still throws:

            groovy.lang.MissingMethodException: No signature of method: java.lang.Boolean.size() is applicable for argument types: () values: []
            Possible solutions: is(java.lang.Object), find(), find(groovy.lang.Closure), with(groovy.lang.Closure), sleep(long), use([Ljava.lang.Object;)
            	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
            	at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
            	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:15)
            	at WorkflowScript.flow(WorkflowScript:52)
            	at WorkflowScript.run(WorkflowScript:91)
            	at ___cps.transform___(Native Method)
            	...
            
            

            How can I work around this?

            Thanks

            Show
            eholzwarth Ed Holzwarth added a comment - I'm not understanding the workaround, apparently. I tried adding the NonCPS annotation as follows: @com.cloudbees.groovy.cps.NonCPS def targetsNotYetDeployed = config.deployTargets.findAll{ !it.hasProperty( "deployed" ) } def done = (targetsNotYetDeployed.size() == 0) But this still throws: groovy.lang.MissingMethodException: No signature of method: java.lang. Boolean .size() is applicable for argument types: () values: [] Possible solutions: is(java.lang. Object ), find(), find(groovy.lang.Closure), with(groovy.lang.Closure), sleep( long ), use([Ljava.lang. Object ;) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55) at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:15) at WorkflowScript.flow(WorkflowScript:52) at WorkflowScript.run(WorkflowScript:91) at ___cps.transform___(Native Method) ... How can I work around this? Thanks
            Hide
            jglick Jesse Glick added a comment -

            @NonCPS is a method annotation. You cannot apply it to a local variable definition.

            Show
            jglick Jesse Glick added a comment - @NonCPS is a method annotation. You cannot apply it to a local variable definition.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-plugin/07f36024c20fb900c99de6d02d7d0acef6f62b5a
            Log:
            JENKINS-26481 Creating integration test.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-plugin/07f36024c20fb900c99de6d02d7d0acef6f62b5a Log: JENKINS-26481 Creating integration test.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/scriptsecurity/sandbox/groovy/SandboxInterceptor.java
            src/main/resources/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/generic-whitelist
            src/test/java/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/StaticWhitelistTest.java
            http://jenkins-ci.org/commit/script-security-plugin/db50903978f66acc38e6ba6f679601499e3542bc
            Log:
            JENKINS-26481 When delegating to DefaultGroovyMethods, check the whitelist against that method, but then do not interfere with actual call processing.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/scriptsecurity/sandbox/groovy/SandboxInterceptor.java src/main/resources/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/generic-whitelist src/test/java/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/StaticWhitelistTest.java http://jenkins-ci.org/commit/script-security-plugin/db50903978f66acc38e6ba6f679601499e3542bc Log: JENKINS-26481 When delegating to DefaultGroovyMethods, check the whitelist against that method, but then do not interfere with actual call processing.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/scriptsecurity/sandbox/groovy/SandboxInterceptor.java
            src/main/resources/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/generic-whitelist
            src/test/java/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/StaticWhitelistTest.java
            http://jenkins-ci.org/commit/script-security-plugin/34086a41a6cf6d6479665cdd65ad23d80ae3129f
            Log:
            Merge pull request #50 from jglick/eachClosure-JENKINS-26481

            JENKINS-26481 DefaultGroovyMethods logic error

            Compare: https://github.com/jenkinsci/script-security-plugin/compare/086735210ce2...34086a41a6cf

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/scriptsecurity/sandbox/groovy/SandboxInterceptor.java src/main/resources/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/generic-whitelist src/test/java/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/StaticWhitelistTest.java http://jenkins-ci.org/commit/script-security-plugin/34086a41a6cf6d6479665cdd65ad23d80ae3129f Log: Merge pull request #50 from jglick/eachClosure- JENKINS-26481 JENKINS-26481 DefaultGroovyMethods logic error Compare: https://github.com/jenkinsci/script-security-plugin/compare/086735210ce2...34086a41a6cf
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/pom.xml
            cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-plugin/b356799f09d18b0d9f96c1084354df072b7d656f
            Log:
            JENKINS-26481 With new script-security and groovy-cps releases, eachClosure finally passes.

            Compare: https://github.com/jenkinsci/workflow-plugin/compare/a990468a828e...b356799f09d1

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/pom.xml cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-plugin/b356799f09d18b0d9f96c1084354df072b7d656f Log: JENKINS-26481 With new script-security and groovy-cps releases, eachClosure finally passes. Compare: https://github.com/jenkinsci/workflow-plugin/compare/a990468a828e...b356799f09d1
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/pom.xml
            cps/src/main/java/org/jenkinsci/plugins/workflow/cps/persistence/IteratorHack.java
            cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-plugin/5b102a5b06745b1eeeb7c1305096bf80e37c1a90
            Log:
            Merge pull request #372 from jenkinsci/eachClosure-JENKINS-26481

            JENKINS-26481 JENKINS-27421 Fix ArrayList$Itr, and integration test for Object.each(Closure)

            Compare: https://github.com/jenkinsci/workflow-plugin/compare/e3906483924d...5b102a5b0674

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/pom.xml cps/src/main/java/org/jenkinsci/plugins/workflow/cps/persistence/IteratorHack.java cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-plugin/5b102a5b06745b1eeeb7c1305096bf80e37c1a90 Log: Merge pull request #372 from jenkinsci/eachClosure- JENKINS-26481 JENKINS-26481 JENKINS-27421 Fix ArrayList$Itr, and integration test for Object.each(Closure) Compare: https://github.com/jenkinsci/workflow-plugin/compare/e3906483924d...5b102a5b0674
            Hide
            abayer Andrew Bayer added a comment - - edited

            I assume additional changes will be needed for, say, collect, collectEntries, any, every, find, findAll?

            Show
            abayer Andrew Bayer added a comment - - edited I assume additional changes will be needed for, say, collect, collectEntries, any, every, find, findAll?
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/f9122193c2099f63ce006f63d619ff3874799cd7
            Log:
            JENKINS-26481 Creating integration test.
            Originally-Committed-As: 07f36024c20fb900c99de6d02d7d0acef6f62b5a

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/f9122193c2099f63ce006f63d619ff3874799cd7 Log: JENKINS-26481 Creating integration test. Originally-Committed-As: 07f36024c20fb900c99de6d02d7d0acef6f62b5a
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            cps/pom.xml
            cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/03650953a2c28412f57d0ea6e32aee6a0a937c69
            Log:
            JENKINS-26481 With new script-security and groovy-cps releases, eachClosure finally passes.
            Originally-Committed-As: b356799f09d18b0d9f96c1084354df072b7d656f

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: cps/pom.xml cps/src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/03650953a2c28412f57d0ea6e32aee6a0a937c69 Log: JENKINS-26481 With new script-security and groovy-cps releases, eachClosure finally passes. Originally-Committed-As: b356799f09d18b0d9f96c1084354df072b7d656f
            Hide
            jglick Jesse Glick added a comment -

            I assume additional changes will be needed for…

            Yes, but see issue 7.

            Show
            jglick Jesse Glick added a comment - I assume additional changes will be needed for… Yes, but see issue 7 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            pom.xml
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/b5c85d9f1663b1c34c8e56b656fd0e2cee662869
            Log:
            [FIXED JENKINS-34064] Backing out JENKINS-26481 fix since it broke Pipeline in Jenkins 2.x.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: pom.xml src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/b5c85d9f1663b1c34c8e56b656fd0e2cee662869 Log: [FIXED JENKINS-34064] Backing out JENKINS-26481 fix since it broke Pipeline in Jenkins 2.x.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            pom.xml
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/ca821248fbc4ec1ebfa00b059eb8162af843df76
            Log:
            Merge pull request #2 from jglick/JENKINS-34064-backout

            JENKINS-34064 Backing out JENKINS-26481 fix

            Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/61fb5860ea27...ca821248fbc4

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: pom.xml src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/ca821248fbc4ec1ebfa00b059eb8162af843df76 Log: Merge pull request #2 from jglick/ JENKINS-34064 -backout JENKINS-34064 Backing out JENKINS-26481 fix Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/61fb5860ea27...ca821248fbc4
            Hide
            jglick Jesse Glick added a comment -

            Backed out in workflow-cps 2.1.

            Show
            jglick Jesse Glick added a comment - Backed out in workflow-cps 2.1.
            Hide
            sradi81 Stefan Rademacher added a comment -

            This issue also ocurrs in the current Jenkins 2.0 release candidate

            Show
            sradi81 Stefan Rademacher added a comment - This issue also ocurrs in the current Jenkins 2.0 release candidate
            Hide
            batmat Baptiste Mathus added a comment -

            Stefan Rademacher the Jenkins version is not really the relevant part, can you check the versions of the installed plugins, more precisely those called pipeline-* or workflow-*?

            And even more precisely, the version of the workflow-cps plugin?

            Thanks

            Show
            batmat Baptiste Mathus added a comment - Stefan Rademacher the Jenkins version is not really the relevant part, can you check the versions of the installed plugins, more precisely those called pipeline-* or workflow-*? And even more precisely, the version of the workflow-cps plugin? Thanks
            Hide
            sradi81 Stefan Rademacher added a comment -

            There is no *-cps plugin installed in my Jenkins 2.0 release candidate. That's why I mentioned, that this error also occurs in Jenkins 2.0.
            The other pipeline plugins have the following versions:
            Pipeline 2.0
            Pipeline: API 2.0
            Pipeline: Basic Steps 2.0
            Pipeline: Build Step 2.0
            Pipeline: Groovy 2.1
            Pipeline: Input Step 2.0
            Pipeline: Job 2.1
            Pipeline: Multibranch 2.1
            Pipeline: Nodes and Processes 2.0
            Pipeline: REST API Plugin 1.3
            Pipeline: SCM Step 2.0
            Pipeline: Shared Groovy Libraries 2.0
            Pipeline: Stage Step 2.0
            Pipeline: Stage View Plugin 1.3
            Pipeline: Step API 2.0

            Show
            sradi81 Stefan Rademacher added a comment - There is no *-cps plugin installed in my Jenkins 2.0 release candidate. That's why I mentioned, that this error also occurs in Jenkins 2.0. The other pipeline plugins have the following versions: Pipeline 2.0 Pipeline: API 2.0 Pipeline: Basic Steps 2.0 Pipeline: Build Step 2.0 Pipeline: Groovy 2.1 Pipeline: Input Step 2.0 Pipeline: Job 2.1 Pipeline: Multibranch 2.1 Pipeline: Nodes and Processes 2.0 Pipeline: REST API Plugin 1.3 Pipeline: SCM Step 2.0 Pipeline: Shared Groovy Libraries 2.0 Pipeline: Stage Step 2.0 Pipeline: Stage View Plugin 1.3 Pipeline: Step API 2.0
            Hide
            pranspach Patrick Ranspach added a comment -

            Can we add this to the epic: 2.0: Pipeline as Code? This seems very important to using Groovy in a pipeline as code. It can be easily reproduced and blocks dependable groovy functionality.

            Also relevant labels: 2.0, 2.0-planned, testfest, windows, workflow

            Maybe also related, I'm only able to access closure arguments within returned closures if I copy the argument:

            def test = { name ->
                def path = 'services/' + name
                return { _ ->
                    println(name)
                    // .. do something
                }
            }
            

            will print the same 'name' with this code:

            def options = [];
            ['test', 'test1', 'test2'].each {
                iter ->
                    options << test(iter)
            }
            options.each {it()}
            return options
            

            In a Jenkins Pipeline I have to save the argument, like:

            def test = { arg ->
                def name = arg;
                def path = 'services/' + name
                return { _ ->
                    println(name)
                    // .. do something
                }
            }
            
            Show
            pranspach Patrick Ranspach added a comment - Can we add this to the epic: 2.0: Pipeline as Code? This seems very important to using Groovy in a pipeline as code. It can be easily reproduced and blocks dependable groovy functionality. Also relevant labels: 2.0, 2.0-planned, testfest, windows, workflow Maybe also related, I'm only able to access closure arguments within returned closures if I copy the argument: def test = { name -> def path = 'services/' + name return { _ -> println(name) // .. do something } } will print the same 'name' with this code: def options = []; [ 'test' , 'test1' , 'test2' ].each { iter -> options << test(iter) } options.each {it()} return options In a Jenkins Pipeline I have to save the argument, like: def test = { arg -> def name = arg; def path = 'services/' + name return { _ -> println(name) // .. do something } }
            Hide
            jglick Jesse Glick added a comment -

            Stefan Rademacher the current display name of the workflow-cps plugin is Pipeline: Groovy.

            Patrick Ranspach be assured that developers are well aware of this issue and are struggling to find a fix. Your “maybe related” sounds unrelated to me, perhaps JENKINS-33468.

            Show
            jglick Jesse Glick added a comment - Stefan Rademacher the current display name of the workflow-cps plugin is Pipeline: Groovy . Patrick Ranspach be assured that developers are well aware of this issue and are struggling to find a fix. Your “maybe related” sounds unrelated to me, perhaps JENKINS-33468 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/efeee2b5a86ca2a5f03efd48fc2c1e596c57e2a2
            Log:
            JENKINS-26481 Fail the script meaningfully when it attempts to pass a CPS-transformed closure to a binary method.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/efeee2b5a86ca2a5f03efd48fc2c1e596c57e2a2 Log: JENKINS-26481 Fail the script meaningfully when it attempts to pass a CPS-transformed closure to a binary method.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/6ad4d04e3df379836cfa8e32072007ab8e59c3a7
            Log:
            JENKINS-26481 Limit the error to cases where the binary method actually requested a Closure, and is thus likely to try to call it.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/6ad4d04e3df379836cfa8e32072007ab8e59c3a7 Log: JENKINS-26481 Limit the error to cases where the binary method actually requested a Closure, and is thus likely to try to call it.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/test/java/org/jenkinsci/plugins/workflow/cps/steps/RestartingLoadStepTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/4266f0ce6b1d1eb07fd4c273592d938cf95bfd23
            Log:
            JENKINS-26481 Verifying that we can still pass closures to functions defined in load’d scripts.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/test/java/org/jenkinsci/plugins/workflow/cps/steps/RestartingLoadStepTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/4266f0ce6b1d1eb07fd4c273592d938cf95bfd23 Log: JENKINS-26481 Verifying that we can still pass closures to functions defined in load’d scripts.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            src/test/java/org/jenkinsci/plugins/workflow/cps/steps/RestartingLoadStepTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/004392892aaec46c5d5a541c30e3230a175fcfb2
            Log:
            Merge pull request #15 from jglick/report-JENKINS-26481

            JENKINS-26481 More clearly report problem passing closures to binary methods

            Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/7364fe748099...004392892aae

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java src/test/java/org/jenkinsci/plugins/workflow/cps/steps/RestartingLoadStepTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/004392892aaec46c5d5a541c30e3230a175fcfb2 Log: Merge pull request #15 from jglick/report- JENKINS-26481 JENKINS-26481 More clearly report problem passing closures to binary methods Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/7364fe748099...004392892aae
            Hide
            marcus_phi Marcus Philip added a comment -

            I get stack trace below when I have this code. I assume it's the same problem.

            def keys = []
            keys.add('key1')
            int afterKeyIndex = keys.findIndexOf { it == 'key1' }
            int index = 1 + afterKeyIndex
            

            In some strange way afterKeyIndex is a boolean!

            hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: java.lang.Integer.plus() is applicable for argument types: (java.lang.Boolean) values: [true]
            Possible solutions: plus(java.lang.Number), plus(java.lang.Character), plus(java.lang.String), abs(), is(java.lang.Object), minus(java.lang.Number)
            	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
            	at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
            	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
            	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
            	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:16)
            	at netent.JenkinsWtfListMap.addAfter(file:/var/lib/jenkins/jobs/ngp-ddds-application-master/builds/213/libs/netentNGPCoreLib/src/netent/JenkinsWtfListMap.groovy:35)
            	at BuildConfig.run(/var/lib/jenkins/jobs/ngp-ddds-application-master/builds/213/libs/netentNGPCoreLib/vars/netentPipelineDsl.groovy:110)
            	at WorkflowScript.run(WorkflowScript:23)
            	at netentPipelineDsl.runCD(/var/lib/jenkins/jobs/ngp-ddds-application-master/builds/213/libs/netentNGPCoreLib/vars/netentPipelineDsl.groovy:141)
            	at ___cps.transform___(Native Method)
            	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:48)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
            	at sun.reflect.GeneratedMethodAccessor826.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:497)
            	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
            	at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:33)
            	at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
            	at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:22)
            	at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
            	at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
            	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
            	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
            	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:324)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224)
            	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
            	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            	at java.lang.Thread.run(Thread.java:745)
            

            It seems I can solve this by encapsulating keys.findIndexOf in a NonCPS method.

            Show
            marcus_phi Marcus Philip added a comment - I get stack trace below when I have this code. I assume it's the same problem. def keys = [] keys.add( 'key1' ) int afterKeyIndex = keys.findIndexOf { it == 'key1' } int index = 1 + afterKeyIndex In some strange way afterKeyIndex is a boolean! hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: java.lang.Integer.plus() is applicable for argument types: (java.lang.Boolean) values: [true] Possible solutions: plus(java.lang.Number), plus(java.lang.Character), plus(java.lang.String), abs(), is(java.lang.Object), minus(java.lang.Number) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55) at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:16) at netent.JenkinsWtfListMap.addAfter(file:/var/lib/jenkins/jobs/ngp-ddds-application-master/builds/213/libs/netentNGPCoreLib/src/netent/JenkinsWtfListMap.groovy:35) at BuildConfig.run(/var/lib/jenkins/jobs/ngp-ddds-application-master/builds/213/libs/netentNGPCoreLib/vars/netentPipelineDsl.groovy:110) at WorkflowScript.run(WorkflowScript:23) at netentPipelineDsl.runCD(/var/lib/jenkins/jobs/ngp-ddds-application-master/builds/213/libs/netentNGPCoreLib/vars/netentPipelineDsl.groovy:141) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:48) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor826.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:33) at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30) at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:22) at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55) at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16) at com.cloudbees.groovy.cps.Next.step(Next.java:58) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:324) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) It seems I can solve this by encapsulating keys.findIndexOf in a NonCPS method.
            Hide
            nickolayr Nickolay Rumyantsev added a comment -

            Marcus Philip, you are correct, this is the same issue. From my observations it always return the first result of the closure, in your case it's a boolean result of "it == 'key1'". @NonCPS should help you.

            Show
            nickolayr Nickolay Rumyantsev added a comment - Marcus Philip , you are correct, this is the same issue. From my observations it always return the first result of the closure, in your case it's a boolean result of " it == 'key1' ". @NonCPS should help you.
            Hide
            marcus_phi Marcus Philip added a comment -

            The absolute irony in this is that I actually wrote unit tests for this code, something I always strive to do but often fail to. This time I could do it because I was writing 'shared code' that did not call step functions. The docs state: "At the base level, any valid Groovy code is OK". Well, I don't know about that ...

            Solution: Provide a unit-testing framework that mangles your code the same way that it will be mangled in Jenkins.

            BTW, you really should give names to the different variants of shared libraries so we can reason and talk about them.

            Show
            marcus_phi Marcus Philip added a comment - The absolute irony in this is that I actually wrote unit tests for this code, something I always strive to do but often fail to. This time I could do it because I was writing 'shared code' that did not call step functions. The docs state: "At the base level, any valid Groovy code is OK". Well, I don't know about that ... Solution: Provide a unit-testing framework that mangles your code the same way that it will be mangled in Jenkins. BTW, you really should give names to the different variants of shared libraries so we can reason and talk about them.
            Hide
            seblez Sébastien Varupenne added a comment - - edited

            I have a problem with pipeline. It says it's this problem :

            [Pipeline] // stage
            [Pipeline] stage
            [Pipeline] { (build-modules)
            [Pipeline] }
            [Pipeline] // stage
            [Pipeline] }
            [Pipeline] // node
            [Pipeline] End of Pipeline
            java.lang.UnsupportedOperationException: Calling public static java.util.Map org.codehaus.groovy.runtime.DefaultGroovyMethods.eachWithIndex(java.util.Map,groovy.lang.Closure) on a CPS-transformed closure is not yet supported (JENKINS-26481); encapsulate in a @NonCPS method, or use Java-style loops
            	at org.jenkinsci.plugins.workflow.cps.GroovyClassLoaderWhitelist.checkJenkins26481(GroovyClassLoaderWhitelist.java:90)
            	at org.jenkinsci.plugins.workflow.cps.GroovyClassLoaderWhitelist.permitsStaticMethod(GroovyClassLoaderWhitelist.java:60)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:92)
            	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
            	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
            	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16)
            	at WorkflowScript.run(WorkflowScript:19)
            	at ___cps.transform___(Native Method)
            	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:48)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
            	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
            	at sun.reflect.GeneratedMethodAccessor972.invoke(Unknown Source)
            	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            	at java.lang.reflect.Method.invoke(Method.java:498)
            	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
            	at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
            	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
            	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
            	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
            	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
            	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:163)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:324)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)
            	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224)
            	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:63)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
            	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
            	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            	at java.lang.Thread.run(Thread.java:745)
            Finished: FAILURE
            

            here is my groovy pipeline code :

            node {
                stage('build-modules') {
                    def modules = [ 'FINANCE' : 'std-module-FINANCE-5_N9-4_manuel',
                                    'MOBILITE' : 'std-module-MOBILE-dev_integration-continue'
                                    ]
                    def buildMods = [:]
            
                    modules.eachWithIndex { module , i ->
                        buildMods["mod${i}"] = {
                            println "${module.key} will build ${module.value}"
                        }
                    }
                    parallel buildMods
                }
            }
            
            Show
            seblez Sébastien Varupenne added a comment - - edited I have a problem with pipeline. It says it's this problem : [Pipeline] // stage [Pipeline] stage [Pipeline] { (build-modules) [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline java.lang.UnsupportedOperationException: Calling public static java.util.Map org.codehaus.groovy.runtime.DefaultGroovyMethods.eachWithIndex(java.util.Map,groovy.lang.Closure) on a CPS-transformed closure is not yet supported (JENKINS-26481); encapsulate in a @NonCPS method, or use Java-style loops at org.jenkinsci.plugins.workflow.cps.GroovyClassLoaderWhitelist.checkJenkins26481(GroovyClassLoaderWhitelist.java:90) at org.jenkinsci.plugins.workflow.cps.GroovyClassLoaderWhitelist.permitsStaticMethod(GroovyClassLoaderWhitelist.java:60) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:92) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16) at WorkflowScript.run(WorkflowScript:19) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:48) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor972.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46) at com.cloudbees.groovy.cps.Next.step(Next.java:58) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:324) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:63) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Finished: FAILURE here is my groovy pipeline code : node { stage( 'build-modules' ) { def modules = [ 'FINANCE' : 'std-module-FINANCE-5_N9-4_manuel' , 'MOBILITE' : 'std-module-MOBILE-dev_integration- continue ' ] def buildMods = [:] modules.eachWithIndex { module , i -> buildMods[ "mod${i}" ] = { println "${module.key} will build ${module.value}" } } parallel buildMods } }
            Hide
            jglick Jesse Glick added a comment -

            You cannot use eachWithIndex or similar methods currently, sorry.

            Show
            jglick Jesse Glick added a comment - You cannot use eachWithIndex or similar methods currently, sorry.
            Hide
            jglick Jesse Glick added a comment -

            Idea to be tested: rather than using DGMPatcher, see if the use syntax can override DefaultGroovyMethods, possibly offering a solution for JENKINS-27421 as well. The problem is a bit of a chicken-and-egg: DefaultGroovyMethods.use(Object, Class, Closure) itself cannot be properly invoked.

            Show
            jglick Jesse Glick added a comment - Idea to be tested: rather than using DGMPatcher , see if the use syntax can override DefaultGroovyMethods , possibly offering a solution for JENKINS-27421 as well. The problem is a bit of a chicken-and-egg: DefaultGroovyMethods.use(Object, Class, Closure) itself cannot be properly invoked.
            Hide
            jglick Jesse Glick added a comment -

            Perhaps could have the CPS engine call GroovyCategorySupport private methods on a predefined category. use itself actually cannot work in a CPS transform; it would have to be applied to the whole CPS VM thread. Or perhaps DGMPatcher can be rewritten to use the tricks done in GroovyCategorySupport: DefaultMetaClassInfo.disabledStandardMC, VMPluginFactory.getPlugin().invalidateCallSites(), etc.

            Show
            jglick Jesse Glick added a comment - Perhaps could have the CPS engine call GroovyCategorySupport private methods on a predefined category. use itself actually cannot work in a CPS transform; it would have to be applied to the whole CPS VM thread. Or perhaps DGMPatcher can be rewritten to use the tricks done in GroovyCategorySupport : DefaultMetaClassInfo.disabledStandardMC , VMPluginFactory.getPlugin().invalidateCallSites() , etc.
            Hide
            kenorb kenorb added a comment - - edited

            Same here, I've Multi-line String Parameter (Params) with multi items like: Foo, Bar, Baz

            Then I'm trying to run this code:

            Params.split("\\r?\\n").each { param ->
                print "Param: ${param}"
            }
            

            Then I've got the error:

            java.lang.UnsupportedOperationException: Calling public static java.lang.Object org.codehaus.groovy.runtime.DefaultGroovyMethods.each(java.lang.Object,groovy.lang.Closure) on a CPS-transformed closure is not yet supported (JENKINS-26481); encapsulate in a @NonCPS method, or use Java-style loops

            Show
            kenorb kenorb added a comment - - edited Same here, I've Multi-line String Parameter (Params) with multi items like: Foo, Bar, Baz Then I'm trying to run this code: Params.split( "\\r?\\n" ).each { param -> print "Param: ${param}" } Then I've got the error: java.lang.UnsupportedOperationException: Calling public static java.lang.Object org.codehaus.groovy.runtime.DefaultGroovyMethods.each(java.lang.Object,groovy.lang.Closure) on a CPS-transformed closure is not yet supported ( JENKINS-26481 ); encapsulate in a @NonCPS method, or use Java-style loops
            Hide
            fasterfish Marat S added a comment -

            Is there any workaround for this issue? What is the deadline for solving this issue?

            Show
            fasterfish Marat S added a comment - Is there any workaround for this issue? What is the deadline for solving this issue?
            Hide
            dothebart Wilfried Goesgens added a comment -

            Hi, I ran into this issue when trying to use a regex.

            Since I find the way of groovy implementing a regex replace a bit strange anyways, couldn't it be a good solution to offer a special regex function?

            so currently the only workaround after two years of this bug to get

            str = '\\cygdrive\\c\\c\\workspace\\RELEASE__BuildTest\\outEP\\arangodb-Windows.tar.gz.md5'
            print(str), 
            print('\n')
            print(str.replaceAll(/^\\cygdrive\\(.)/ , \{ full, driveletter -> driveletter + ":" }))
            

            working currently would be

             sh(returnStdout: true, script: 'echo ${str} | sed .... )
            

            ??

            Show
            dothebart Wilfried Goesgens added a comment - Hi, I ran into this issue when trying to use a regex. Since I find the way of groovy implementing a regex replace a bit strange anyways, couldn't it be a good solution to offer a special regex function? so currently the only workaround after two years of this bug to get str = '\\cygdrive\\c\\c\\workspace\\RELEASE__BuildTest\\outEP\\arangodb-Windows.tar.gz.md5' print(str), print( '\n' ) print(str.replaceAll(/^\\cygdrive\\(.)/ , \{ full, driveletter -> driveletter + ":" })) working currently would be  sh(returnStdout: true , script: 'echo ${str} | sed .... ) ??
            Hide
            jglick Jesse Glick added a comment -

            Wilfried Goesgens wrap in @NonCPS.

            Show
            jglick Jesse Glick added a comment - Wilfried Goesgens wrap in @NonCPS .
            Hide
            abayer Andrew Bayer added a comment -

            Once https://github.com/cloudbees/groovy-cps/pull/51 lands, and https://github.com/jenkinsci/jenkins/pull/2814 makes it into an LTS, we can bump workflow-cps to use that LTS (and therefore Groovy 2.4.11). I'm getting all the ducks in a row for this by writing up patched CPS methods for as many of the problematic DefaultGroovyMethods as I can, so I should have that ready to go as soon as said LTS is available.

            Show
            abayer Andrew Bayer added a comment - Once https://github.com/cloudbees/groovy-cps/pull/51 lands, and https://github.com/jenkinsci/jenkins/pull/2814 makes it into an LTS, we can bump workflow-cps to use that LTS (and therefore Groovy 2.4.11). I'm getting all the ducks in a row for this by writing up patched CPS methods for as many of the problematic DefaultGroovyMethods as I can, so I should have that ready to go as soon as said LTS is available.
            Hide
            markl_lagendijk Mark Lagendijk added a comment -

            Andrew Bayer Groovy 2.4.11 was released in Jenkins 2.61.

            Any progress on this issue?

            Show
            markl_lagendijk Mark Lagendijk added a comment - Andrew Bayer Groovy 2.4.11 was released in Jenkins 2.61. Any progress on this issue?
            Hide
            jglick Jesse Glick added a comment -

            Mark Lagendijk yes there is progress, though it does not depend on Groovy 2.4.11.

            Show
            jglick Jesse Glick added a comment - Mark Lagendijk yes there is progress, though it does not depend on Groovy 2.4.11.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/java/org/jenkinsci/plugins/workflow/cps/CpsThread.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/ab9a154de1f91c6b99b75c577898079950ed3ac1
            Log:
            JENKINS-26481 Prototype of using GroovyCategorySupport to invoke CpsDefaultGroovyMethods without hacks like DGMPatcher.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/workflow/cps/CpsThread.java src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/ab9a154de1f91c6b99b75c577898079950ed3ac1 Log: JENKINS-26481 Prototype of using GroovyCategorySupport to invoke CpsDefaultGroovyMethods without hacks like DGMPatcher.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            pom.xml
            src/main/java/org/jenkinsci/plugins/workflow/cps/CpsThread.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/CpsWhitelist.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java
            src/main/java/org/jenkinsci/plugins/workflow/cps/persistence/IteratorHack.java
            src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java
            src/test/java/org/jenkinsci/plugins/workflow/cps/SnippetizerTest.java
            src/test/java/org/jenkinsci/plugins/workflow/cps/persistence/IteratorHackTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/3ec591db4bca053eb70534a92583dcd5b52bf6e5
            Log:
            Merge pull request #124 from jglick/GroovyCategorySupport-JENKINS-26481

            JENKINS-26481 JENKINS-27421 Use GroovyCategorySupport to invoke CpsDefaultGroovyMethods (w/o DGMPatcher) & IteratorHack

            Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/b48af161645f...3ec591db4bca

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/cps/CpsThread.java src/main/java/org/jenkinsci/plugins/workflow/cps/CpsWhitelist.java src/main/java/org/jenkinsci/plugins/workflow/cps/GroovyClassLoaderWhitelist.java src/main/java/org/jenkinsci/plugins/workflow/cps/SandboxContinuable.java src/main/java/org/jenkinsci/plugins/workflow/cps/persistence/IteratorHack.java src/test/java/org/jenkinsci/plugins/workflow/SerializationTest.java src/test/java/org/jenkinsci/plugins/workflow/cps/SnippetizerTest.java src/test/java/org/jenkinsci/plugins/workflow/cps/persistence/IteratorHackTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/3ec591db4bca053eb70534a92583dcd5b52bf6e5 Log: Merge pull request #124 from jglick/GroovyCategorySupport- JENKINS-26481 JENKINS-26481 JENKINS-27421 Use GroovyCategorySupport to invoke CpsDefaultGroovyMethods (w/o DGMPatcher) & IteratorHack Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/b48af161645f...3ec591db4bca
            Hide
            jbochenski Jakub Bochenski added a comment -

            I still have issues with this as described in JENKINS-46747

            Show
            jbochenski Jakub Bochenski added a comment - I still have issues with this as described in JENKINS-46747
            Hide
            jbochenski Jakub Bochenski added a comment -

            Another way this still fails: JENKINS-46749

            Show
            jbochenski Jakub Bochenski added a comment - Another way this still fails: JENKINS-46749
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            content/blog/2017/08/2017-09-08-enumerators-in-pipeline.adoc
            http://jenkins-ci.org/commit/jenkins.io/3ed6f8445ded8ddafec2e0e01b71ced9af0c6e7d
            Log:
            Add a nice little blog post about the despised JENKINS-26481

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/blog/2017/08/2017-09-08-enumerators-in-pipeline.adoc http://jenkins-ci.org/commit/jenkins.io/3ed6f8445ded8ddafec2e0e01b71ced9af0c6e7d Log: Add a nice little blog post about the despised JENKINS-26481
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            content/blog/2017/08/2017-09-08-enumerators-in-pipeline.adoc
            http://jenkins-ci.org/commit/jenkins.io/89792a0c8e0a3850f95ec5fe24bbc89f962fb7ed
            Log:
            Merge pull request #1120 from rtyler/enumerators

            Add a nice little blog post about the despised JENKINS-26481

            Compare: https://github.com/jenkins-infra/jenkins.io/compare/1c5922c91f13...89792a0c8e0a

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/blog/2017/08/2017-09-08-enumerators-in-pipeline.adoc http://jenkins-ci.org/commit/jenkins.io/89792a0c8e0a3850f95ec5fe24bbc89f962fb7ed Log: Merge pull request #1120 from rtyler/enumerators Add a nice little blog post about the despised JENKINS-26481 Compare: https://github.com/jenkins-infra/jenkins.io/compare/1c5922c91f13...89792a0c8e0a
            Hide
            lucasc Lucas Cimon added a comment -

            Thank you for fixing this !

            Show
            lucasc Lucas Cimon added a comment - Thank you for fixing this !
            Hide
            murme Ulad Muraveika added a comment -

            Guys, it will be cool if you update documentation on https://jenkins.io/doc/pipeline/examples/ page, because bug was fixed and doc right now is confusing novice users

            >Due to limitations in Workflow - i.e., JENKINS-26481 - it's not really possible to use Groovy closures or syntax that depends on closures, so you can't do the Groovy standard of using .collectEntries on a list and generating the steps as values for the resulting entries. You also can't use the standard Java syntax for For loops - i.e., "for (String s: strings)" - and instead have to use old school counter-based for loops.

            Show
            murme Ulad Muraveika added a comment - Guys, it will be cool if you update documentation on https://jenkins.io/doc/pipeline/examples/ page, because bug was fixed and doc right now is confusing novice users >Due to limitations in Workflow - i.e., JENKINS-26481 - it's not really possible to use Groovy closures or syntax that depends on closures, so you can't do the Groovy standard of using .collectEntries on a list and generating the steps as values for the resulting entries. You also can't use the standard Java syntax for For loops - i.e., "for (String s: strings)" - and instead have to use old school counter-based for loops.
            Hide
            jglick Jesse Glick added a comment -

            Ulad Muraveika good catch. Could you file an issue for this in the WEBSITE project? Of course if you have the time to do it, we welcome PRs updating documentation. (In the case of code samples, this means testing that the proposed syntax actually works in current releases.)

            Show
            jglick Jesse Glick added a comment - Ulad Muraveika good catch. Could you file an issue for this in the WEBSITE project? Of course if you have the time to do it, we welcome PRs updating documentation. (In the case of code samples, this means testing that the proposed syntax actually works in current releases.)
            Hide
            batmat Baptiste Mathus added a comment -

            Ulad Muraveika or anyone having the time to fix this, please note that https://jenkins.io is fully backed by https://github.com/jenkins-infra/jenkins.io, so a PR there is the only thing needed to fix it.

            (Side note: not only guys can fix this issue )

            Show
            batmat Baptiste Mathus added a comment - Ulad Muraveika or anyone having the time to fix this, please note that https://jenkins.io is fully backed by https://github.com/jenkins-infra/jenkins.io , so a PR there is the only thing needed to fix it. (Side note: not only guys can fix this issue )
            Hide
            murme Ulad Muraveika added a comment -

            Baptiste Mathus, I'll do this, because I had funny time with digging around NonSerializable exceptions and have created small documentation file with all such cases for my team. Now it is time to share it

            Thanks for additional info and reminder!

            Show
            murme Ulad Muraveika added a comment - Baptiste Mathus , I'll do this, because I had funny time with digging around NonSerializable exceptions and have created small documentation file with all such cases for my team. Now it is time to share it Thanks for additional info and reminder!
            Hide
            ilatypov Ilguiz Latypov added a comment - - edited

            Another limit is still annoying because it silently returns from a NonCPS method calling a CPS method.

             

                @NonCPS
                public static String readCurrentVersion(String fileContent) {
                    def xml = new XmlSlurper().parseText(deBOM(fileContent))
                    String retval = "${xml.metadata.version}"
                    xml = null
                    return retval
                }
            
                // Omitted @NonCPS here
                public static CharSequence deBOM(CharSequence s) {
                    [...]
                    return s
                }
            

            Calling readCurrentVersion from the above unexpectedly returned the entire fileContent.  This is probably another unexpected behaviour but seems worth mentioning in the documentation.

             

            Show
            ilatypov Ilguiz Latypov added a comment - - edited Another limit is still annoying because it silently returns from a NonCPS method calling a CPS method.   @NonCPS public static String readCurrentVersion( String fileContent) { def xml = new XmlSlurper().parseText(deBOM(fileContent)) String retval = "${xml.metadata.version}" xml = null return retval } // Omitted @NonCPS here public static CharSequence deBOM(CharSequence s) { [...] return s } Calling readCurrentVersion from the above unexpectedly returned the entire fileContent.  This is probably another unexpected behaviour but seems worth mentioning in the documentation.  
            Hide
            jglick Jesse Glick added a comment -

            Ilguiz Latypov this ought to have been addressed (in the form of a warning) by JENKINS-31314.

            Show
            jglick Jesse Glick added a comment - Ilguiz Latypov this ought to have been addressed (in the form of a warning) by JENKINS-31314 .

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              dtschan Daniel Tschan
              Votes:
              108 Vote for this issue
              Watchers:
              136 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: