Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62659

Intermittent NotSerializableException when iterating over a LinkedList in a Pipeline

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      workflow-cps 2.81

      Description

      Originally reported as https://github.com/cloudbees/groovy-cps/issues/105.

      Using a LinkedList in a Groovy for in loop in a Pipeline sometimes (but not always!) causes the following NotSerializableException:

      an exception which occurred:
          in field com.cloudbees.groovy.cps.impl.ForInLoopBlock$ContinuationImpl.itr
          in object com.cloudbees.groovy.cps.impl.ForInLoopBlock$ContinuationImpl@4365f5c7
          in field com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.target
          in object com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl@701a709d
          in field com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv.continue_
          in object com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv@44fc4585
          in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
          in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@323c441a
          in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
          in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@5e61e8aa
          in field com.cloudbees.groovy.cps.impl.CallEnv.caller
          in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@7e82466a
          in field com.cloudbees.groovy.cps.Continuable.e
          in object com.cloudbees.groovy.cps.Continuable@125a5d42
          in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
          in object org.jenkinsci.plugins.workflow.cps.CpsThread@3eda4dde
          in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
          in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56f14ac9
          in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56f14ac9
      Caused: java.io.NotSerializableException: java.util.LinkedList$ListItr
          ...
      

      Here is a simple Pipeline that can reproduce the problem (make sure you are using the MAX_SURVIVABILITY durability level):

      def col = new LinkedList<>([1, 2, 3])
      for (Integer i in col) {
        sleep(time: 500, unit: 'MILLISECONDS') // Force the Pipeline to be serialized.
      }
      

      My diagnosis of the issue can be found in this and earlier comments. When resolving this call internally, sometimes Groovy treats it as a call to List.iterator, but sometimes it treats it as a call to Deque.iterator() (presumably because LinkedList implements both interfaces and there is some nondeterministic behavior somewhere). The method that Groovy chooses changes which IteratorHack category methods are valid (the class of the selected method much match the first parameter of the category method), so adding IteratorHack.iterator(Deque) fixes the issue.

      This probably also affects other types, and maybe we need to add other methods to IteratorHack as well.

      I am not sure about the root cause of the nondeterminism, and I was not able to reproduce it using a standalone Groovy script. See this comment for addition details. Notably, running a JenkinsRule-based reproduction test in a loop inside of an @Test method showed that the test either always fails or always passes in the same JVM instance, so maybe the nondeterminism is caused by something like ordering of methods in Java's reflection APIs, which I think is undefined but consistent during the lifetime of a single JVM instance.

        Attachments

          Issue Links

            Activity

            Hide
            dnusbaum Devin Nusbaum added a comment -

            A fix for this issue was just released in Pipeline: Groovy plugin 2.81. If you are still seeing the issue, but for a different type, please open a new ticket with the full stack trace; it is probably a distinct issue.

            Show
            dnusbaum Devin Nusbaum added a comment - A fix for this issue was just released in Pipeline: Groovy plugin 2.81. If you are still seeing the issue, but for a different type, please open a new ticket with the full stack trace; it is probably a distinct issue.

              People

              Assignee:
              dnusbaum Devin Nusbaum
              Reporter:
              dnusbaum Devin Nusbaum
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: