[JENKINS-23712] Exceptions in multi-conf jobs and concurrents executions

Type: Bug
Resolution: Fixed
Priority: Major
Component/s: cluster-stats-plugin
Labels:
None
Environment:
Jenkins LT 1.544.3, ClusterStats 0.4.4

Similar Issues:
Powered by SuggestiMate

Show

I get more and more exception like this in Jenkins log :

WARNING: RunListener failed
java.lang.RuntimeException: Failed to serialize org.zeroturnaround.stats.ClusterStatisticsPlugin#statsData for class org.zeroturnaround.stats.ClusterStatisticsPlugin
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:214)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:182)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:167)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:108)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
	at hudson.XmlFile.write(XmlFile.java:178)
	at hudson.Plugin.save(Plugin.java:250)
	at org.zeroturnaround.stats.ClusterStatisticsPlugin.maybeSave(ClusterStatisticsPlugin.java:57)
	at org.zeroturnaround.stats.StartCompleteListener.onCompleted(StartCompleteListener.java:50)
	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:199)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:231)
Caused by: java.lang.RuntimeException: Failed to serialize org.zeroturnaround.stats.model.StatsData#runStats for class org.zeroturnaround.stats.model.StatsData
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:214)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:182)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:167)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:108)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:223)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:210)
	... 21 more
Caused by: java.util.ConcurrentModificationException
	at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
	at java.util.AbstractList$Itr.next(AbstractList.java:343)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:223)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:210)
	... 30 more
Jul 8, 2014 3:39:27 PM hudson.model.listeners.RunListener report
WARNING: RunListener failed
java.lang.RuntimeException: Failed to serialize org.zeroturnaround.stats.ClusterStatisticsPlugin#statsData for class org.zeroturnaround.stats.ClusterStatisticsPlugin
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:214)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:182)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:167)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:108)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
	at hudson.XmlFile.write(XmlFile.java:178)
	at hudson.Plugin.save(Plugin.java:250)
	at org.zeroturnaround.stats.ClusterStatisticsPlugin.maybeSave(ClusterStatisticsPlugin.java:57)
	at org.zeroturnaround.stats.StartCompleteListener.onCompleted(StartCompleteListener.java:50)
	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:199)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:231)
Caused by: java.lang.RuntimeException: Failed to serialize org.zeroturnaround.stats.model.StatsData#runStats for class org.zeroturnaround.stats.model.StatsData
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:214)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:182)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:167)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:108)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:223)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:210)
	... 21 more
Caused by: java.util.ConcurrentModificationException
	at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
	at java.util.AbstractList$Itr.next(AbstractList.java:343)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:223)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:210)
	... 30 more
Jul 8, 2014 3:39:27 PM hudson.model.listeners.RunListener report
WARNING: RunListener failed
java.lang.RuntimeException: Failed to serialize org.zeroturnaround.stats.ClusterStatisticsPlugin#statsData for class org.zeroturnaround.stats.ClusterStatisticsPlugin
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:214)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:182)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:167)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:108)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
	at hudson.XmlFile.write(XmlFile.java:178)
	at hudson.Plugin.save(Plugin.java:250)
	at org.zeroturnaround.stats.ClusterStatisticsPlugin.maybeSave(ClusterStatisticsPlugin.java:57)
	at org.zeroturnaround.stats.StartCompleteListener.onCompleted(StartCompleteListener.java:50)
	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:199)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:231)
Caused by: java.lang.RuntimeException: Failed to serialize org.zeroturnaround.stats.model.StatsData#runStats for class org.zeroturnaround.stats.model.StatsData
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:214)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:182)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:167)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:108)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:223)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:210)
	... 21 more
Caused by: java.util.ConcurrentModificationException
	at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
	at java.util.AbstractList$Itr.next(AbstractList.java:343)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)

These exceptions appears when multi-conf jobs with build stuff sent to 10 slaves at the same time.

Side effect, some builds failed on some slaves (never the same) and randomly ie :

;; connection timed out; trying next origin
;; connection timed out; no servers could be reached
Build step 'Execute shell' marked build as failure

This plugin is still maintained ?
Anyone with similar issues ?

Shannon Kerr added a comment - 2014-08-22 18:05

I think this person maintains this plugin.

Shannon Kerr added a comment - 2014-08-22 18:05 I think this person maintains this plugin.

SCM/JIRA link daemon added a comment - 2014-08-25 07:49

Code changed in jenkins
User: Toomas Romer
Path:
src/main/java/org/zeroturnaround/stats/ClusterStatisticsPlugin.java
src/main/java/org/zeroturnaround/stats/model/StatsData.java
http://jenkins-ci.org/commit/cluster-stats/cd77bff3a1aee1d1da084377ff2c638e80393d83
Log:
~~JENKINS-23712~~ Syncrhonizing file saving

Compare: https://github.com/jenkinsci/cluster-stats/compare/fca409901858...cd77bff3a1ae

SCM/JIRA link daemon added a comment - 2014-08-25 07:49 Code changed in jenkins User: Toomas Romer Path: src/main/java/org/zeroturnaround/stats/ClusterStatisticsPlugin.java src/main/java/org/zeroturnaround/stats/model/StatsData.java http://jenkins-ci.org/commit/cluster-stats/cd77bff3a1aee1d1da084377ff2c638e80393d83 Log: JENKINS-23712 Syncrhonizing file saving Compare: https://github.com/jenkinsci/cluster-stats/compare/fca409901858...cd77bff3a1ae

toomasr added a comment - 2014-08-25 08:21 - edited

I did some changes:

Had to bump the dependency version to 1.557
- I thought 1.520 is enough but then I stumbled on ~~JENKINS-23712~~
Also synchronized the file saving that should fix the problem
Release 0.4.5 version

Let me know if this solves your issues.

toomasr added a comment - 2014-08-25 08:21 - edited I did some changes: Had to bump the dependency version to 1.557 I thought 1.520 is enough but then I stumbled on JENKINS-23712 Also synchronized the file saving that should fix the problem Release 0.4.5 version Let me know if this solves your issues.

Jenkins

Details

Description

Attachments

Activity

Collapse comment: Shannon Kerr added a comment - 2014-08-22 18:05

Expand comment: Shannon Kerr added a comment - 2014-08-22 18:05

Collapse comment: SCM/JIRA link daemon added a comment - 2014-08-25 07:49

Expand comment: SCM/JIRA link daemon added a comment - 2014-08-25 07:49

Collapse comment: toomasr added a comment - 2014-08-25 08:21, Edited by toomasr - 2014-08-25 08:21

Expand comment: toomasr added a comment - 2014-08-25 08:21, Edited by toomasr - 2014-08-25 08:21

People

Dates