Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49732

java.util.LinkedHashMap$Entry not serializable in Pipeline

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • workflow-cps-plugin
    • None
    • Jenkins 2.108
      latest plugins
      Master: Windows 2008 R2 x64, 2.
      Slave: Windows 10 Pro x64

       

      def remote_watch_urls = [
          './branches': 'my url/branches', 
          './releases': 'my url/releases'
          ]
      for ( loc in remote_watch_urls )
      {
          echo "remote_watch_urls server: ${loc.key}: ${loc.value}"
      }
      stage ('Get extension API versions')
      {
          node('BUILD_WIN_64') {
              ws("w/${env.JOB_NAME}".replace('%', '_')) {
                  stage("Checkout") {
                      for ( loc in remote_watch_urls )
                      {
                          echo "remote_watch_urls slave: ${loc.key}: ${loc.value}"
                          checkout changelog: false, scm: [$class: 'SubversionSCM', additionalCredentials: [], excludedCommitMessages: '', excludedRegions: '', excludedRevprop: '', excludedUsers: '', filterChangelog: false, ignoreDirPropChanges: false, includedRegions: '', locations: [[credentialsId: 'my credentialsid', depthOption: 'infinity', ignoreExternalsOption: true, local: loc.key, remote: loc.value]], quietOperation: true, workspaceUpdater: [$class: 'UpdateWithCleanUpdater']]
                      }
                  }
              }
         }
      }
      

      This is a minial script for repro. The checkout step fails and I'm not sure why. The echo steps work fine. There's more steps and stages after the checkout steps that's not included, but actually nothing prior to the posted script.

      [BFA] Scanning build for known causes...
       [BFA] No failure causes found
       [BFA] Done. 2s
       an exception which occurred:
       in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
       in object com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv@e92a85
       in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
       in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@1fc9156
       in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
       in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@190fed5
       in field com.cloudbees.groovy.cps.impl.CallEnv.caller
       in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@1864fef
       in field com.cloudbees.groovy.cps.Continuable.e
       in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@ad9bff
       in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
       in object org.jenkinsci.plugins.workflow.cps.CpsThread@1c261a6
       in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
       in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@bdabe9
       in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@bdabe9
       Caused: java.io.NotSerializableException: java.util.LinkedHashMap$Entry
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
       at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
       at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
       at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
       at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
       at java.io.ObjectOutputStream.writeObject(Unknown Source)
       at java.util.HashMap.internalWriteEntries(Unknown Source)
       at java.util.HashMap.writeObject(Unknown Source)
       at sun.reflect.GeneratedMethodAccessor167.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:273)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
       at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
       at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
       at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
       at java.io.ObjectOutputStream.writeObject(Unknown Source)
       at java.util.TreeMap.writeObject(Unknown Source)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:273)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
       at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
       at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
       at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
       at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
       at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
       at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:467)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:443)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:430)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:367)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:82)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:243)
       at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:231)
       at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
       at java.util.concurrent.FutureTask.run(Unknown Source)
       at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
       at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
       at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
       at java.util.concurrent.FutureTask.run(Unknown Source)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
       at java.lang.Thread.run(Unknown Source)
       Finished: FAILURE

          [JENKINS-49732] java.util.LinkedHashMap$Entry not serializable in Pipeline

          Oleg Nenashev added a comment -

          "Caused: java.io.NotSerializableException: java.util.LinkedHashMap$Entry"

          Oleg Nenashev added a comment - "Caused: java.io.NotSerializableException: java.util.LinkedHashMap$Entry"

          Andrew Bayer added a comment -

          So the problem, as oleg_nenashev noted, is a serialization failure for java.util.LinkedHashMap$Entry. You can reproduce this minimally as follows:

          def remote_watch_urls = [
              './branches': 'my url/branches', 
              './releases': 'my url/releases'
              ]
          for (foo in remote_watch_urls) {
            sleep 1
            echo "foo is ${foo}"
          }
          

          The sleep will trigger Pipeline serialization and things blow up there. I'm not sure if we'll ever be able to actually fix that particular syntax, but there's a trivial workaround:

          remote_watch_urls.each { foo ->
            sleep 1
            echo "foo is ${foo}"
          }
          

          ...should work with no errors.

          Andrew Bayer added a comment - So the problem, as oleg_nenashev noted, is a serialization failure for java.util.LinkedHashMap$Entry . You can reproduce this minimally as follows: def remote_watch_urls = [ './branches' : 'my url/branches' , './releases' : 'my url/releases' ] for (foo in remote_watch_urls) { sleep 1 echo "foo is ${foo}" } The sleep will trigger Pipeline serialization and things blow up there. I'm not sure if we'll ever be able to actually fix that particular syntax, but there's a trivial workaround: remote_watch_urls.each { foo -> sleep 1 echo "foo is ${foo}" } ...should work with no errors.

          Edgars Batna added a comment -

          The workaround helps.

          Even if not soon, it should be looked at, get fixed or get a better error message. The latter would be a time-saver.

          Edgars Batna added a comment - The workaround helps. Even if not soon, it should be looked at, get fixed or get a better error message. The latter would be a time-saver.

          Ulad Muraveika added a comment - - edited

          Hi abayer,

          It will be very helpful, if you explain why your workaround with groovy closure will work, because I've tried to debug similar code and can see java.util.LinkedHashMap$Entry variables as well.

          Does serialization works in different way for variables in closures?

          My answer: yes, it is working different and it can be used as workaround per fixed https://issues.jenkins-ci.org/browse/JENKINS-26481

          Thanks in advance!

           

          Ulad Muraveika added a comment - - edited Hi abayer , It will be very helpful, if you explain why your workaround with groovy closure will work, because I've tried to debug similar code and can see java.util.LinkedHashMap$Entry variables as well. Does serialization works in different way for variables in closures? My answer: yes, it is working different and it can be used as workaround per fixed https://issues.jenkins-ci.org/browse/JENKINS-26481 Thanks in advance!  

          Fabian Holler added a comment -

          Does it mean the workaround using `.each` instead does not work anymore since JENKINS-26481?

          Fabian Holler added a comment - Does it mean the workaround using `.each` instead does not work anymore since JENKINS-26481 ?

          fho, please use my small cheat sheet below for all fine points in the pipelines code. Quick answer - you can use .each due to closed JENKINS-26481(https://issues.jenkins-ci.org/browse/JENKINS-26481)

          Example of code Can it be used in the pipeline code?
          Iterator from plain C-style for-loop Yes
          Iterator from Groovy each - JENKINS-26481(https://issues.jenkins-ci.org/browse/JENKINS-26481) Yes
          Java 5-style for loop for (data_type item : collection) - JENKINS-27421(https://issues.jenkins-ci.org/browse/JENKINS-27421) Yes
          *WARNING - if you are using : Java 5-style for loop for (data_type item : collection) for ANY implementation of Map interface - and want to work with Map Entries you will have java.io.NotSerializableException - java.util.LinkedHashMap$Entry - JENKINS-49732(https://issues.jenkins-ci.org/browse/JENKINS-49732).
          Fix will be to use groovy each data.each closure or use visibly entrySet with Java for loop for (def foo : data.entrySet())
          No
          Any code connected with a regex like code - find, replace, split, groovy syntax sugar =~ and ==~, because all of them have non Serializable java.util.regex.Matcher inside No
          JSONSlupper with groovy.json.internal.LazyMap inside. Please use groovy.json.JsonSlurperClassic with java.util.HashMap instead or better use [builtin Jenkins step ](https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace) - readJSON No

          Ulad Muraveika added a comment - fho , please use my small cheat sheet below for all fine points in the pipelines code. Quick answer - you can use .each due to closed JENKINS-26481 ( https://issues.jenkins-ci.org/browse/JENKINS-26481 ) Example of code Can it be used in the pipeline code? Iterator from plain C-style for-loop Yes Iterator from Groovy each - JENKINS-26481 ( https://issues.jenkins-ci.org/browse/JENKINS-26481 ) Yes Java 5-style for loop for (data_type item : collection) - JENKINS-27421 ( https://issues.jenkins-ci.org/browse/JENKINS-27421 ) Yes * WARNING - if you are using : Java 5-style for loop for (data_type item : collection) for ANY implementation of Map interface - and want to work with Map Entries you will have java.io.NotSerializableException - java.util.LinkedHashMap$Entry - JENKINS-49732 ( https://issues.jenkins-ci.org/browse/JENKINS-49732 ). Fix will be to use groovy each data.each closure or use visibly entrySet with Java for loop for (def foo : data.entrySet()) No Any code connected with a regex like code - find, replace, split, groovy syntax sugar =~ and ==~, because all of them have non Serializable java.util.regex.Matcher inside No JSONSlupper with groovy.json.internal.LazyMap inside. Please use groovy.json.JsonSlurperClassic with java.util.HashMap instead or better use [builtin Jenkins step ] ( https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace ) - readJSON No

          Fabian Holler added a comment -

          murme thanks a lot, that is very useful!

          Fabian Holler added a comment - murme thanks a lot, that is very useful!

          junwei ning added a comment -

          I encountered the same problem, how can I solve it?

           

          this is my code:

          // var/code_pull.groovy

          def call(Map branch_mapping) {
          def group_mapping=[
          'robokit_3rd':'robokit',
          'robokit_framework':'robokit'
          ]
          echo "$branch_mapping"
          println branch_mapping
          println group_mapping
          for (item in branch_mapping) {
          def git_group = group_mapping.get(item.key)
          def project = item.key
          def branch = item.value
          println git_group
          println item.key
          println item.value
          checkout([$class: 'GitSCM', branches: [[name: "${branch}"]],
          doGenerateSubmoduleConfigurations: false,
          extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: "${project}"]],
          submoduleCfg: [],
          userRemoteConfigs: [[credentialsId: 'gitlab', url: "gitlab@git.narwel.com:${git_group}/${project}.git"]]]
          )

          }
          }

          junwei ning added a comment - I encountered the same problem, how can I solve it?   this is my code: // var/code_pull.groovy def call(Map branch_mapping) { def group_mapping=[ 'robokit_3rd':'robokit', 'robokit_framework':'robokit' ] echo "$branch_mapping" println branch_mapping println group_mapping for (item in branch_mapping) { def git_group = group_mapping.get(item.key) def project = item.key def branch = item.value println git_group println item.key println item.value checkout([$class: 'GitSCM', branches: [ [name: "${branch}"] ], doGenerateSubmoduleConfigurations: false, extensions: [ [$class: 'RelativeTargetDirectory', relativeTargetDir: "${project}"] ], submoduleCfg: [], userRemoteConfigs: [ [credentialsId: 'gitlab', url: "gitlab@git.narwel.com:${git_group}/${project}.git"] ]] ) } }

          Dor added a comment - - edited

          Hey, I encountered the same thing, and I advise you ningjunwei to use what is suggested in this comment: https://issues.jenkins.io/browse/JENKINS-49732?focusedCommentId=330459&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-330459 

          It helped me with it too.

          Dor added a comment - - edited Hey, I encountered the same thing, and I advise you ningjunwei  to use what is suggested in this comment: https://issues.jenkins.io/browse/JENKINS-49732?focusedCommentId=330459&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-330459   It helped me with it too.

            Unassigned Unassigned
            gl1koz3 Edgars Batna
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: