Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44924

pipeline groovy script - Sort a list with custom comparator or closure not sorting

    XMLWordPrintable

Details

    Description

       

      When using groovy script in the Pipeline Plugin, sorting a list using closure or a custom comparator does not work anymore.

       

      Steps to reproduce:

      1. create new item of type Pipeline
      2. In the Pipeline script add the following code
      #!groovy

assert ["aa","bb","cc"] == ["aa","cc","bb"].sort { a, b -> a <=> b }
      1. Click Save
      2. Click Build Now
      3. Check the failed build:

      [Pipeline] End of Pipeline
      hudson.remoting.ProxyException: Assertion failed:

      assert ["aa","bb","cc"] == ["aa","cc","bb"].sort { a, b -> a <=> b }

      at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:404)
      at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:650)
      at com.cloudbees.groovy.cps.impl.AssertBlock$ContinuationImpl.fail(AssertBlock.java:47)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
      at com.cloudbees.groovy.cps.Next.step(Next.java:83)
      at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:173)
      at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:162)
      at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
      at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
      at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:162)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:19)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:35)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:32)
      at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:32)
      at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:330)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
      at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE

      Expected result is a sorted list:

      [aa, bb, cc]

       

       

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-26481 Mishandling of binary methods accepting Closure

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-50343 ComposedClosure not working with List.each

          • Major - Major loss of function.
          • Open

          Activity

            Ascending order - Click to sort in descending order
            View 19 older comments
            lucasc Lucas Cimon added a comment - - edited

            I found bit of helpful context about this issue : https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Callingnon-CPS-transformedmethodswithCPS-transformedarguments

            Edit: captain obvious... I did not notice the link to this exact issue on the wiki

            Here is the workaround I used to sort a list of Map objects, based on a String field:

            pipeline {
    stages {
        stage("Test") {
            steps {
                script {
                    def propList = [[name: 'B'], [name: 'A']]
                    sortByField(propList, 'name')
                    propList.eachWithIndex { entry, i ->
                        println "${i}: ${entry}"
                    }
                }
            }
        }
    }
}
@NonCPS
static sortByField(list, fieldName) {
    list.sort{ it[fieldName] }
}
            lucasc Lucas Cimon added a comment - - edited I found bit of helpful context about this issue : https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Callingnon-CPS-transformedmethodswithCPS-transformedarguments Edit: captain obvious... I did not notice the link to this exact issue on the wiki Here is the workaround I used to sort a list of Map objects, based on a String field: pipeline { stages { stage( "Test" ) { steps { script { def propList = [[name: 'B' ], [name: 'A' ]] sortByField(propList, 'name' ) propList.eachWithIndex { entry, i -> println "${i}: ${entry}" } } } } } } @NonCPS static sortByField(list, fieldName) { list.sort{ it[fieldName] } }
            ianfixes Ian Katz added a comment -

            If I could have viably fixed this, I would have already - the transformation done to Groovy code to be able to be durable makes a lot of things more or less impossible to get working correctly. We've dealt with as many of them as we could without having to basically write our own implementation of all of Groovy, but there are still edge cases (sorry, but this is a comparative edge case) where we just can't fix them.

            If the sort function can't work under any circumstances, shouldn't the correct behavior be to have it throw some exception rather than just pretend to work?  (Emphasis for meaning, not for tone.)

            I lost about a day's work here trying permutations of the following documented descriptions of array sort:

            I would have greatly preferred an exception that said "We're sorry, but this is what it is."  Instead I assumed that I wasn't supplying the correct arguments, mutating instead of returning a value, etc.  That was a very unpleasant rabbit hole.

            Can you talk more about what makes function impossible to support?  For example, is it related to a threaded library or some other implementation detail that prevents the built-in function from every being translated to CPS?  Is it a lack of support for the "spaceship operator"?  Or is it something inherent to the sort algorithm that would make it fail even if I tried to implement my own array sort function by hand?

            ianfixes Ian Katz added a comment - If I could have viably fixed this, I would have already - the transformation done to Groovy code to be able to be durable makes a lot of things more or less impossible to get working correctly. We've dealt with as many of them as we could without having to basically write our own implementation of all of Groovy, but there are still edge cases (sorry, but this is a comparative edge case) where we just can't fix them. If the sort function can't work under any circumstances, shouldn't the correct behavior be to have it throw some exception rather than just pretend to work?  (Emphasis for meaning, not for tone.) I lost about a day's work here trying permutations of the following documented descriptions of array sort: http://docs.groovy-lang.org/next/html/documentation/working-with-collections.html#_sorting https://mrhaki.blogspot.com/2015/03/groovy-goodness-new-methods-to-sort-and.html https://mrhaki.blogspot.com/2011/09/groovy-goodness-sort-or-remove.html https://stackoverflow.com/a/20386474/2063546 I would have greatly preferred an exception that said "We're sorry, but this is what it is."  Instead I assumed that I wasn't supplying the correct arguments, mutating instead of returning a value, etc.  That was a very unpleasant rabbit hole. Can you talk more about what makes function impossible to support?  For example, is it related to a threaded library or some other implementation detail that prevents the built-in function from every being translated to CPS?  Is it a lack of support for the "spaceship operator"?  Or is it something inherent to the sort algorithm that would make it fail even if I tried to implement my own array sort function by hand?
            jglick Jesse Glick added a comment -

            If you are using a reasonably recent version of workflow-cps it should have printed a warning with a link to a wiki page explaining this class of issue and workarounds.

            jglick Jesse Glick added a comment - If you are using a reasonably recent version of workflow-cps it should have printed a warning with a link to a wiki page explaining this class of issue and workarounds.
            ianfixes Ian Katz added a comment -

            I'm on 2.190.2 of Jenkins, with BlueOcean 1.21.0. Not sure how to find workflow-cps version.

            I didn't see a warning, but my point is that it should produce an error, not a warning. What is the justification for allowing me to call a function that you know in advance will not do its job?

            ianfixes Ian Katz added a comment - I'm on 2.190.2 of Jenkins, with BlueOcean 1.21.0. Not sure how to find workflow-cps version. I didn't see a warning, but my point is that it should produce an error, not a warning. What is the justification for allowing me to call a function that you know in advance will not do its job?
            jglick Jesse Glick added a comment -

            /pluginManager/installed; text console log in classic view, not Blue Ocean; and because the detector has false positives (it is very complicated).

            jglick Jesse Glick added a comment - /pluginManager/installed ; text console log in classic view, not Blue Ocean; and because the detector has false positives (it is very complicated).

            People

              Unassigned Unassigned
              persal79 Per Salomonsson
              Votes:
              29 Vote for this issue
              Watchers:
              43 Start watching this issue

              Dates

                Created:
                Updated: