Critical ParallelStepTest.invisibleParallelBranch observed to fail with:
java.lang.RuntimeException: Failed to serialize org.jenkinsci.plugins.workflow.job.WorkflowRun#logsToCopy for class org.jenkinsci.plugins.workflow.job.WorkflowRun at ... Caused by: java.util.ConcurrentModificationException: null at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394) at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:413) at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:412) at com.thoughtworks.xstream.converters.collections.MapConverter.marshal(MapConverter.java:75) at ... at hudson.XmlFile.write(XmlFile.java:178) at hudson.model.Run.save(Run.java:1915) at org.jenkinsci.plugins.workflow.support.steps.input.InputStepExecution.postSettlement(InputStepExecution.java:196) at org.jenkinsci.plugins.workflow.support.steps.input.InputStepExecution.proceed(InputStepExecution.java:149) at org.jenkinsci.plugins.workflow.steps.parallel.ParallelStepTest$9.evaluate(ParallelStepTest.java:360) at ...
Normally Run.save would be called at the end of copyLogs, when the state is consistent, but this is coming from another thread. Possibly needs to be some kind of concurrent map.
- is blocking
-
JENKINS-30055 Simple script with "long" loop created unloadable flow
-
- Resolved
-
[JENKINS-27057] CME in WorkflowRun.logsToCopy
| Labels | Original: threads xstream | New: testing threads xstream |
| Status | Original: Open [ 1 ] | New: In Progress [ 3 ] |
| Link |
New:
This issue is blocking |
Tomasz Szuba
made changes -
| Priority | Original: Minor [ 4 ] | New: Critical [ 2 ] |
Tomasz Szuba
made changes -
| Comment |
[ Recently same CME started to appear on my local instance, but with different stacktrace. It is coming up about once in 5 runs. {code:none} org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step my-step failed at org.jenkinsci.plugins.workflow.cps.steps.ParallelStep$ResultHandler$Callback.checkAllDone(ParallelStep.java:153) at org.jenkinsci.plugins.workflow.cps.steps.ParallelStep$ResultHandler$Callback.onFailure(ParallelStep.java:119) at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$FailureAdapter.receive(CpsBodyExecution.java:294) at com.cloudbees.groovy.cps.impl.ThrowBlock$1.receive(ThrowBlock.java:68) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:58) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:145) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:274) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:74) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:183) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:181) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-1 failed ... 21 more Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-2 failed ... 21 more Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-3 failed ... 21 more Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-4 failed ... 21 more Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-5 failed ... 21 more Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-6 failed ... 21 more Caused by: org.jenkinsci.plugins.workflow.cps.steps.ParallelStepException: Parallel step step-7 failed at org.jenkinsci.plugins.workflow.cps.steps.ParallelStep$ResultHandler$Callback.checkAllDone(ParallelStep.java:153) at org.jenkinsci.plugins.workflow.cps.steps.ParallelStep$ResultHandler$Callback.onSuccess(ParallelStep.java:110) at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution$SuccessAdapter.receive(CpsBodyExecution.java:311) at com.cloudbees.groovy.cps.Outcome.resumeFrom(Outcome.java:73) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:137) ... 14 more Caused by: java.lang.RuntimeException: Failed to serialize org.jenkinsci.plugins.workflow.job.WorkflowRun#logsToCopy for class org.jenkinsci.plugins.workflow.job.WorkflowRun at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:215) at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:183) at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138) at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:168) at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:109) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43) at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82) at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37) at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026) at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015) at com.thoughtworks.xstream.XStream.toXML(XStream.java:988) at hudson.XmlFile.write(XmlFile.java:178) at hudson.model.Run.save(Run.java:1899) at hudson.util.PersistedList.onModified(PersistedList.java:173) at hudson.util.PersistedList._onModified(PersistedList.java:181) at hudson.util.PersistedList.add(PersistedList.java:72) at org.jenkinsci.plugins.workflow.job.WorkflowRun.onCheckout(WorkflowRun.java:623) at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$1100(WorkflowRun.java:107) at org.jenkinsci.plugins.workflow.job.WorkflowRun$SCMListenerImpl.onCheckout(WorkflowRun.java:773) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:123) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:49) at hudson.security.ACL.impersonate(ACL.java:213) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:47) ... 5 more Caused by: java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:711) at java.util.LinkedHashMap$LinkedEntryIterator.next(LinkedHashMap.java:744) at java.util.LinkedHashMap$LinkedEntryIterator.next(LinkedHashMap.java:742) at com.thoughtworks.xstream.converters.collections.MapConverter.marshal(MapConverter.java:75) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84) at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:224) at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:211) ... 31 more {code} ] |
Code changed in jenkins
User: Jesse Glick
Path:
job/src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java
http://jenkins-ci.org/commit/workflow-plugin/6140881717bf937ecb42bf77ec11dcc208bedd0c
Log:
[FIXED JENKINS-27057] Made logsToCopy a concurrent map, since WorkflowRun.save can be called from various threads.
The mutations are all guarded by WorkflowRun.completed, so we just need to allow serialization to see a snapshot.
(There was already a race condition in the case of abrupt shutdown, but at worst these should result in duplicated log text.)
| Resolution | New: Fixed [ 1 ] | |
| Status | Original: In Progress [ 3 ] | New: Resolved [ 5 ] |
Code changed in jenkins
User: Jesse Glick
Path:
aggregator/src/test/java/org/jenkinsci/plugins/workflow/steps/EchoStepTest.java
job/src/main/java/org/jenkinsci/plugins/workflow/job/WorkflowRun.java
http://jenkins-ci.org/commit/workflow-plugin/6de84edb0dd337b2179cb3f1bca1de396c10907d
Log:
JENKINS-27057 Correcting mistake in 6140881717bf937ecb42bf77ec11dcc208bedd0c.
Log text was being doubly copied.
Observed to fail more consistently when applying a fix for
JENKINS-30055.