[JENKINS-23514] Deadlock in Xstream at Jenkins startup

Type: Bug
Resolution: Duplicate
Priority: Major
Component/s: core
Labels:
None
Environment:
Jenkins ver. 1.554.2.2 (Jenkins Enterprise by CloudBees 14.05)
MacOS 10.9.3
java version "1.7.0_55"

Similar Issues:
Powered by SuggestiMate

Show

Deadlock in com.thoughtworks.xstream.mapper.AnnotationMapper when Jenkins loads folders (see thread dump fragment, "www", "mobile" and "admin"are Jenkins folders).

Deadlock occur systematically when the stdout and stderr are routed to the shell console. The workaround is to route stdout and stderr to a file.

Attached:

jenkins-startup.log: jenkins stdout & stderr
deadlock.log: threaddump highlighting the dead lock in XStream

Java stack information for the threads listed above:
===================================================
"Loading job www":
	at java.lang.Class.initAnnotationsIfNecessary(Class.java:3175)
	- waiting to lock <0x00000007e22586b0> (a java.lang.Class for hudson.model.Job)
	at java.lang.Class.getAnnotation(Class.java:3137)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$UnprocessedTypesSet.add(AnnotationMapper.java:625)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$UnprocessedTypesSet.add(AnnotationMapper.java:610)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.processAnnotations(AnnotationMapper.java:181)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.getLocalConverter(AnnotationMapper.java:153)
	at com.thoughtworks.xstream.mapper.MapperWrapper.getLocalConverter(MapperWrapper.java:99)
	at hudson.util.xstream.MapperDelegate.getLocalConverter(MapperDelegate.java:112)
	at com.thoughtworks.xstream.mapper.MapperWrapper.getLocalConverter(MapperWrapper.java:99)
	at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:348)
	at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:289)
	at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:228)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189)
	at hudson.util.XStream2.unmarshal(XStream2.java:113)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
	at hudson.XmlFile.read(XmlFile.java:142)
	at hudson.model.Items.load(Items.java:275)
	at hudson.model.ItemGroupMixIn.loadChildren(ItemGroupMixIn.java:105)
	at com.cloudbees.hudson.plugins.folder.Folder.onLoad(Folder.java:238)
	at hudson.model.Items.load(Items.java:276)
	at jenkins.model.Jenkins$18.run(Jenkins.java:2588)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$7.runTask(Jenkins.java:899)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
"Loading job admin":
	at java.lang.Class.initAnnotationsIfNecessary(Class.java:3175)
	- waiting to lock <0x00000007e1dce000> (a java.lang.Class for hudson.model.Run)
	at java.lang.Class.getAnnotation(Class.java:3137)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$UnprocessedTypesSet.add(AnnotationMapper.java:625)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$UnprocessedTypesSet.add(AnnotationMapper.java:610)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$1.add(AnnotationMapper.java:250)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$1.add(AnnotationMapper.java:245)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.addParametrizedTypes(AnnotationMapper.java:279)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.processTypes(AnnotationMapper.java:204)
	- locked <0x00000007e22586b0> (a java.lang.Class for hudson.model.Job)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.processAnnotations(AnnotationMapper.java:182)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.defaultImplementationOf(AnnotationMapper.java:141)
	at com.thoughtworks.xstream.mapper.MapperWrapper.defaultImplementationOf(MapperWrapper.java:46)
	at hudson.util.xstream.MapperDelegate.defaultImplementationOf(MapperDelegate.java:59)
	at com.thoughtworks.xstream.mapper.MapperWrapper.defaultImplementationOf(MapperWrapper.java:46)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:54)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189)
	at hudson.util.XStream2.unmarshal(XStream2.java:113)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
	at hudson.XmlFile.read(XmlFile.java:142)
	at hudson.model.Items.load(Items.java:275)
	at hudson.model.ItemGroupMixIn.loadChildren(ItemGroupMixIn.java:105)
	at com.cloudbees.hudson.plugins.folder.Folder.onLoad(Folder.java:238)
	at hudson.model.Items.load(Items.java:276)
	at jenkins.model.Jenkins$18.run(Jenkins.java:2588)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$7.runTask(Jenkins.java:899)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
"Loading job mobile":
	at java.lang.Class.initAnnotationsIfNecessary(Class.java:3175)
	- waiting to lock <0x00000007e22586b0> (a java.lang.Class for hudson.model.Job)
	at java.lang.Class.getAnnotation(Class.java:3137)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$UnprocessedTypesSet.add(AnnotationMapper.java:625)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$UnprocessedTypesSet.add(AnnotationMapper.java:610)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$1.add(AnnotationMapper.java:250)
	at com.thoughtworks.xstream.mapper.AnnotationMapper$1.add(AnnotationMapper.java:245)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.addParametrizedTypes(AnnotationMapper.java:279)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.processTypes(AnnotationMapper.java:204)
	- locked <0x00000007e1dce000> (a java.lang.Class for hudson.model.Run)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.processAnnotations(AnnotationMapper.java:182)
	at com.thoughtworks.xstream.mapper.AnnotationMapper.defaultImplementationOf(AnnotationMapper.java:141)
	at com.thoughtworks.xstream.mapper.MapperWrapper.defaultImplementationOf(MapperWrapper.java:46)
	at hudson.util.xstream.MapperDelegate.defaultImplementationOf(MapperDelegate.java:59)
	at com.thoughtworks.xstream.mapper.MapperWrapper.defaultImplementationOf(MapperWrapper.java:46)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:54)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189)
	at hudson.util.XStream2.unmarshal(XStream2.java:113)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
	at hudson.XmlFile.read(XmlFile.java:142)
	at hudson.model.Items.load(Items.java:275)
	at hudson.model.ItemGroupMixIn.loadChildren(ItemGroupMixIn.java:105)
	at com.cloudbees.hudson.plugins.folder.Folder.onLoad(Folder.java:238)
	at hudson.model.Items.load(Items.java:276)
	at jenkins.model.Jenkins$18.run(Jenkins.java:2588)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
	at jenkins.model.Jenkins$7.runTask(Jenkins.java:899)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

Found 1 deadlock.

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

deadlock.log
64 kB
2014-06-20 15:01
jebc-startup-logs-deadlock.log
4 kB
2014-06-20 15:01

is duplicated by

JENKINS-23455 Deadlock during startup

Resolved

is related to

JENKINS-19561 Unsafe & inefficient concurrency in XStream

Fixed but Unreleased

links to

CloudBees Internal DEV-1578

Daniel Beck added a comment - 2014-07-05 23:13

Another workaround might be to set the system property hudson.model.Hudson.parallelLoad to false.

Daniel Beck added a comment - 2014-07-05 23:13 Another workaround might be to set the system property hudson.model.Hudson.parallelLoad to false.

Daniel Beck added a comment - 2014-07-05 23:16

This may even be duplicate of ~~JENKINS-19561~~, given Jesse's comment there, but I'm not sure.

Daniel Beck added a comment - 2014-07-05 23:16 This may even be duplicate of JENKINS-19561 , given Jesse's comment there, but I'm not sure.

Jesse Glick added a comment - 2014-07-18 15:24

Yeah it is a duplicate—this is one possible symptom.

Jesse Glick added a comment - 2014-07-18 15:24 Yeah it is a duplicate—this is one possible symptom.

Details

Description

Attachments

Attachments

Issue Links

Activity

Collapse comment: Daniel Beck added a comment - 2014-07-05 23:13

Expand comment: Daniel Beck added a comment - 2014-07-05 23:13

Collapse comment: Daniel Beck added a comment - 2014-07-05 23:16

Expand comment: Daniel Beck added a comment - 2014-07-05 23:16

Collapse comment: Jesse Glick added a comment - 2014-07-18 15:24

Expand comment: Jesse Glick added a comment - 2014-07-18 15:24

People

Dates