[JENKINS-47071] BooleanClosureWrapper isn't serializable, causing .every and friends to fail

Type: Bug
Resolution: Fixed
Priority: Minor
Component/s: workflow-cps-plugin
Labels:
None
Environment:
workflow-cps 2.40, groovy-cps 1.19

Similar Issues:
Powered by SuggestiMate

Show

If you do something like

['a', 'b'].every { sleep 1; it != null }

in a Pipeline script, you get the always-fun serialization error:

WARNING	o.j.p.w.cps.CpsThreadGroup#saveProgramIfPossible: program state save failed
an exception which occurred:
	in field locals
	in field parent
	in field parent
	in field caller
	in field parent
	in field parent
	in field caller
	in field e
	in field program
	in field threads
	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@65d813fb
Caused: java.io.NotSerializableException: org.codehaus.groovy.runtime.callsite.BooleanClosureWrapper
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
	at java.util.HashMap.internalWriteEntries(HashMap.java:1777)
	at java.util.HashMap.writeObject(HashMap.java:1354)
	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
	at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
	at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
	at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
	at java.util.TreeMap.writeObject(TreeMap.java:2433)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
	at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
	at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
	at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:140)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:458)
Caused: java.io.IOException: Failed to persist /var/folders/pr/24nv8g910wg8vr4b4c33q34r0000gn/T/jenkinsTests.tmp/jenkins6845617186175380474test/jobs/p/builds/1/program.dat
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:472)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:434)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:422)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:362)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:744)

This is because a bunch of the DGM methods (specifically any, count, every, findAll, findIndexOf, findIndexValues, findLastIndexOf, find, removeAll, retainAll, and split) use a BooleanClosureWrapper to actually evaluate their closures. And neither BooleanClosureWrapper nor BooleanReturningMethodInvoker (a field in BooleanClosureWrapper is of this class) are serializable. So as soon as a serialization attempt hits, bam, it falls over.

On the upside, I think I can fix this by just adding new methods to Builder that provides the same behavior as BooleanClosureWrapper.call and BooleanClosureWrapper.callForMap.

is duplicated by

JENKINS-47064 Can no longer load pom file within 'when/expression' block on 1.2

Closed

relates to

JENKINS-47064 Can no longer load pom file within 'when/expression' block on 1.2

Closed

links to

groovy-cps PR #75

workflow-cps PR #177

Laurent Malvert added a comment - 2017-09-24 21:20 - edited

I may be wrong, but I think I'm also getting hit by this bug in a Jenkinsfile where I use a sh() call's return status as part of a when condition's expression, e.g.:

stage("reset") {
  when { anyOf {
    expression { params.RESET_WORKSPACE }
    expression { sh(
        script:"./scripts/build/stage-reset-check.sh ${params.VERSION}",
        returnStatus:true
      ) }
  } }
  steps {
    sh './scripts/build/stage-reset.sh'
  }
}

Laurent Malvert added a comment - 2017-09-24 21:20 - edited I may be wrong, but I think I'm also getting hit by this bug in a Jenkinsfile where I use a sh() call's return status as part of a when condition's expression , e.g.: stage( "reset" ) { when { anyOf { expression { params.RESET_WORKSPACE } expression { sh( script: "./scripts/build/stage-reset-check.sh ${params.VERSION}" , returnStatus: true ) } } } steps { sh './scripts/build/stage-reset.sh' } }

Laurent Malvert added a comment - 2017-09-24 22:13

And indeed the workaround suggested in https://issues.jenkins-ci.org/browse/JENKINS-47064 by nv035674 worked (discard when conditions, write conditions as a guard in a script block):

// temp fix using script block, see:
//   https://issues.jenkins-ci.org/browse/JENKINS-47071
//   https://issues.jenkins-ci.org/browse/JENKINS-47064
//   when { anyOf {
//     expression { params.RESET_WORKSPACE }
//     expression { sh(
//       script:"./scripts/build/stage-reset-check.sh ${params.VERSION}",
//       returnStatus:true
//     ) }
//   } }
  steps {
    script {
      if (params.RESET_WORKSPACE || sh(
        script:"./scripts/build/stage-reset-check.sh ${params.VERSION}",
        returnStatus:true
      )) {
        sh "./do/stuff.sh"
      }
    }
  }

Only satisfactory as a temp fix, but will do until workflow-cps:2.41

Laurent Malvert added a comment - 2017-09-24 22:13 And indeed the workaround suggested in https://issues.jenkins-ci.org/browse/JENKINS-47064 by nv035674 worked (discard when conditions, write conditions as a guard in a script block): // temp fix using script block, see: // https://issues.jenkins-ci.org/browse/JENKINS-47071 // https://issues.jenkins-ci.org/browse/JENKINS-47064 // when { anyOf { // expression { params.RESET_WORKSPACE } // expression { sh( // script: "./scripts/build/stage-reset-check.sh ${params.VERSION}" , // returnStatus: true // ) } // } } steps { script { if (params.RESET_WORKSPACE || sh( script: "./scripts/build/stage-reset-check.sh ${params.VERSION}" , returnStatus: true )) { sh "./ do /stuff.sh" } } } Only satisfactory as a temp fix, but will do until workflow-cps:2.41

Andrew Bayer added a comment - 2017-09-25 00:01

Yup, that's the same root cause.

Andrew Bayer added a comment - 2017-09-25 00:01 Yup, that's the same root cause.

Laurent Malvert added a comment - 2017-09-25 06:06 - edited

I'd consider it slightly higher than minor though, considering this has the potential to break pipelines, and the error message is fairly tricky to troubleshoot to figure out the workaround.

I see a PR has been sent (but is pending a valid CI build). I suppose workflow-cps:2.41can be expected somewhat shortly?

Laurent Malvert added a comment - 2017-09-25 06:06 - edited I'd consider it slightly higher than minor though, considering this has the potential to break pipelines, and the error message is fairly tricky to troubleshoot to figure out the workaround. I see a PR has been sent (but is pending a valid CI build). I suppose workflow-cps:2.41can be expected somewhat shortly?

Chris Stylianou added a comment - 2017-09-28 12:14

@abayer Any idea when an update will be release, as this issue is currently breaking all of our pipeline builds

Chris Stylianou added a comment - 2017-09-28 12:14 @ abayer Any idea when an update will be release, as this issue is currently breaking all of our pipeline builds

Andrew Bayer added a comment - 2017-09-28 12:20

I believe it'll be released today - svanoort, are you still planning to do that workflow-cps release?

Andrew Bayer added a comment - 2017-09-28 12:20 I believe it'll be released today - svanoort , are you still planning to do that workflow-cps release?

Sam Van Oort added a comment - 2017-09-28 13:50

abayer Yes – I'm testing since there's a lot of changes in this version and trying to get one community contribution in before cutting.

Sam Van Oort added a comment - 2017-09-28 13:50 abayer Yes – I'm testing since there's a lot of changes in this version and trying to get one community contribution in before cutting.

Stefan Thurnherr added a comment - 2017-10-01 14:01

abayer svanoort Thanks for solving this! Can confirm that "Pipeline: Groovy" 2.41 together with Declarative/Model API/Stage Tags Metadata plugins 1.2.1 works fine for us.

Stefan Thurnherr added a comment - 2017-10-01 14:01 abayer svanoort Thanks for solving this! Can confirm that "Pipeline: Groovy" 2.41 together with Declarative/Model API/Stage Tags Metadata plugins 1.2.1 works fine for us.

Joe George added a comment - 2018-03-07 16:07 - edited

I'm still getting this error in switch statements.

switch(foobarstring) {
    case { it.contains("foo") }:
        echo "foo"
        break
    case { it.contains("bar") }:
        echo "bar"
        break
}

Causing:

Caused: java.io.NotSerializableException: org.codehaus.groovy.runtime.callsite.BooleanClosureWrapper

Pipeline: Groovy 2.35

Pipeline: Declarative 1.2.7

Joe George added a comment - 2018-03-07 16:07 - edited I'm still getting this error in switch statements. switch (foobarstring) { case { it.contains( "foo" ) }: echo "foo" break case { it.contains( "bar" ) }: echo "bar" break } Causing: Caused: java.io.NotSerializableException: org.codehaus.groovy.runtime.callsite.BooleanClosureWrapper Pipeline: Groovy 2.35 Pipeline: Declarative 1.2.7

Andrew Bayer added a comment - 2018-03-07 16:15

externl - that looks like a different issue specific to closures in case statements. Could you open a new ticket for that in the workflow-cps-plugin component? Thanks!

Andrew Bayer added a comment - 2018-03-07 16:15 externl - that looks like a different issue specific to closures in case statements. Could you open a new ticket for that in the workflow-cps-plugin component? Thanks!

Assignee:: Andrew Bayer

Reporter:: Andrew Bayer

Votes:: 0 Vote for this issue

Watchers:: 8 Start watching this issue

Created:: 2017-09-22 19:53

Updated:: 2018-03-07 16:15

Resolved:: 2017-09-22 21:12

Jenkins

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Laurent Malvert added a comment - 2017-09-24 21:20, Edited by Laurent Malvert - 2017-09-24 21:20

Expand comment: Laurent Malvert added a comment - 2017-09-24 21:20, Edited by Laurent Malvert - 2017-09-24 21:20

Collapse comment: Laurent Malvert added a comment - 2017-09-24 22:13

Expand comment: Laurent Malvert added a comment - 2017-09-24 22:13

Collapse comment: Andrew Bayer added a comment - 2017-09-25 00:01

Expand comment: Andrew Bayer added a comment - 2017-09-25 00:01

Collapse comment: Laurent Malvert added a comment - 2017-09-25 06:06, Edited by Laurent Malvert - 2017-09-25 06:07

Expand comment: Laurent Malvert added a comment - 2017-09-25 06:06, Edited by Laurent Malvert - 2017-09-25 06:07

Collapse comment: Chris Stylianou added a comment - 2017-09-28 12:14

Expand comment: Chris Stylianou added a comment - 2017-09-28 12:14

Collapse comment: Andrew Bayer added a comment - 2017-09-28 12:20

Expand comment: Andrew Bayer added a comment - 2017-09-28 12:20

Collapse comment: Sam Van Oort added a comment - 2017-09-28 13:50

Expand comment: Sam Van Oort added a comment - 2017-09-28 13:50

Collapse comment: Stefan Thurnherr added a comment - 2017-10-01 14:01

Expand comment: Stefan Thurnherr added a comment - 2017-10-01 14:01

Collapse comment: Joe George added a comment - 2018-03-07 16:07, Edited by Joe George - 2018-03-07 16:12

Expand comment: Joe George added a comment - 2018-03-07 16:07, Edited by Joe George - 2018-03-07 16:12

Collapse comment: Andrew Bayer added a comment - 2018-03-07 16:15

Expand comment: Andrew Bayer added a comment - 2018-03-07 16:15

People

Dates