Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47363

Pipeline cps fails when creating array with more than 250 elements

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • workflow-cps-plugin
    • None
    • Jenkins 2.73.1
    • Pipeline - December

      This code: 

      b=[
      0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,
      25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,
      50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,
      75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,
      100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,
      125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,
      150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,
      175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,
      200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,
      225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,
      250 
      ]
      echo b.size().toString()
      

      Produces this error:

      Started by user L. Jenkins
      java.lang.NoSuchMethodError: org.codehaus.groovy.runtime.ArrayUtil.createArray(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)[Ljava/lang/Object;
      	at WorkflowScript.___cps___77(WorkflowScript)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
      	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrapNoCoerce.invoke(StaticMetaMethodSite.java:151)
      	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
      	at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:186)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedStaticCall(Checker.java:188)
      	at org.kohsuke.groovy.sandbox.impl.Checker$checkedStaticCall.callStatic(Unknown Source)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:222)
      	at WorkflowScript.<clinit>(WorkflowScript)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at java.lang.Class.newInstance(Class.java:442)
      	at org.codehaus.groovy.runtime.InvokerHelper.createScript(InvokerHelper.java:434)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:129)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:123)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:517)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:480)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:269)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:419)
      Finished: FAILURE
      Page generated: Oct 9, 2017 8:20:34 PM UTCREST APIJenkins ver. 2.82
      

      Commenting out item "250" (which is the 251st item in the list), causes the pipeline to succeed.

       

      There are any number of work-arounds for this, such as creating an empty array and then using array.addAll() on a number of smaller arrays, but it is completely arbitrary and the error output is obtuse and confusing for a user just instantiating an array.  Further, this causes the pipeline to fail on startup, meaning there is no line number information to help the user find the source of the error. 

       

          [JENKINS-47363] Pipeline cps fails when creating array with more than 250 elements

          Mohammed Omar added a comment -

          Hi All,

          I am also facing this issue. Do we have any workarounds listed any where? Do we have any solution ?

          Mohammed Omar added a comment - Hi All, I am also facing this issue. Do we have any workarounds listed any where? Do we have any solution ?

          Andrew Bayer added a comment -

          Huh. ArrayUtil.createArray is a weird one. Need to figure out how it's getting called...

          Andrew Bayer added a comment - Huh. ArrayUtil.createArray is a weird one. Need to figure out how it's getting called...

          Andrew Bayer added a comment -

          So the underlying problem is that Groovy won't actually let you do more than 250 varargs. i.e., try the following in Groovy outside of Jenkins and it'll fail in the same way:

          def b= makeList(
          0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,
          25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,
          50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,
          75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,
          100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,
          125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,
          150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,
          175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,
          200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,
          225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,
          250
          )
          def makeList(Object... args) {
            def foo = []
            foo.addAll(args)
            return foo  
          }
          

          Now, why is this relevant? Because Groovy's call-site logic does some black magic for bytecode optimizations in method lookups, and that magic uses ArrayUtil.createArray for determining if there's a specific signature that's valid rather than just going with varargs. And ArrayUtil.createArray has implementations for 1 to 250 elements...and no more. So things blow up in the CPS compile phase. I'll see if I can find a workaround.

          Andrew Bayer added a comment - So the underlying problem is that Groovy won't actually let you do more than 250 varargs. i.e., try the following in Groovy outside of Jenkins and it'll fail in the same way: def b= makeList( 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24, 25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49, 50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74, 75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99, 100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124, 125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149, 150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174, 175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199, 200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224, 225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249, 250 ) def makeList( Object ... args) { def foo = [] foo.addAll(args) return foo } Now, why is this relevant? Because Groovy's call-site logic does some black magic for bytecode optimizations in method lookups, and that magic uses ArrayUtil.createArray for determining if there's a specific signature that's valid rather than just going with varargs. And ArrayUtil.createArray has implementations for 1 to 250 elements...and no more. So things blow up in the CPS compile phase. I'll see if I can find a workaround.

          Andrew Bayer added a comment -

          I've got a very tentative PR for working around this up at https://github.com/cloudbees/groovy-cps/pull/78, but I'm not at all convinced it's a good idea. It's...weird. Probably would be better to somehow throw a compilation error if a list expression of >250 elements or a map expression of >125 entries (same underlying issue in both cases) is encountered.

          Andrew Bayer added a comment - I've got a very tentative PR for working around this up at https://github.com/cloudbees/groovy-cps/pull/78 , but I'm not at all convinced it's a good idea. It's...weird. Probably would be better to somehow throw a compilation error if a list expression of >250 elements or a map expression of >125 entries (same underlying issue in both cases) is encountered.

          Andrew Bayer added a comment -

          ...and a new PR is up at https://github.com/cloudbees/groovy-cps/pull/79 that I think I like better. It gives a compilation error on encountering a list expression of >250 elements or a map expression of >125 entries. I think that's probably a better approach than the hackish workaround in the first PR.

          Andrew Bayer added a comment - ...and a new PR is up at https://github.com/cloudbees/groovy-cps/pull/79 that I think I like better. It gives a compilation error on encountering a list expression of >250 elements or a map expression of >125 entries. I think that's probably a better approach than the hackish workaround in the first PR.

          Mohammed Omar added a comment - - edited

          abayer

          this  https://github.com/cloudbees/groovy-cps/pull/79 just seems to be for proper messaging , when it fail. I am looking for a solution where I should be able to declare more than 250 elements or any other way to have all those declarations.

          Mohammed Omar added a comment - - edited abayer this  https://github.com/cloudbees/groovy-cps/pull/79 just seems to be for proper messaging , when it fail. I am looking for a solution where I should be able to declare more than 250 elements or any other way to have all those declarations.

          Andrew Bayer added a comment -

          mohaomar - my investigation showed that it was nontrivially complicated, without an obvious non-hacky solution, so we're opting to go with better error messaging for now. You can work around this problem simply by declaring your lists in multiple parts and adding them together.

          Andrew Bayer added a comment - mohaomar - my investigation showed that it was nontrivially complicated, without an obvious non-hacky solution, so we're opting to go with better error messaging for now. You can work around this problem simply by declaring your lists in multiple parts and adding them together.

          Andrew Bayer added a comment -

          Merged in groovy-cps, need to cut a release and then do a workflow-cps PR as well.

          Andrew Bayer added a comment - Merged in groovy-cps , need to cut a release and then do a workflow-cps PR as well.

          Andrew Bayer added a comment -

          This is minor enough that it's probably not worth cutting a workflow-cps release on its own, so we'll wait to pick this up when groovy-cps next has something meriting a release.

          Andrew Bayer added a comment - This is minor enough that it's probably not worth cutting a workflow-cps release on its own, so we'll wait to pick this up when groovy-cps next has something meriting a release.

          Jesse Glick added a comment -

          workflow-cps PR 198 picks this up.

          Jesse Glick added a comment - workflow-cps PR 198 picks this up.

            abayer Andrew Bayer
            bitwiseman Liam Newman
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: