Arnaud Héritier added a comment - 2012-11-19 10:01 - edited Same thing on my side : https://gist.github.com/4109911 (1.491)

Elmar Deppe added a comment - 2012-11-20 16:23 Got the same problem with 1.489 (running in a tomcat 6). Here is the relevant segment from the thread dump. Found one Java-level deadlock: ============================= "Handling GET /jenkins//api/xml : http-8080-37": waiting to lock monitor 0x00000000103b4a88 (object 0x0000000780a10130, a hudson.model.RunMap), which is held by "Handling GET /jenkins/ : http-8080-6" "Handling GET /jenkins/ : http-8080-6": waiting to lock monitor 0x0000000011f37498 (object 0x00000007871608c8, a hudson.maven.MavenBuild), which is held by "Executor #0 for master : executing DSODP_branch_DEV-1.3 #144" "Executor #0 for master : executing DSODP_branch_DEV-1.3 #144": waiting to lock monitor 0x00000000103b4a88 (object 0x0000000780a10130, a hudson.model.RunMap), which is held by "Handling GET /jenkins/ : http-8080-6" Java stack information for the threads listed above: =================================================== "Handling GET /jenkins//api/xml : http-8080-37": at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638) - waiting to lock <0x0000000780a10130> (a hudson.model.RunMap) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621) at jenkins.model.lazy.AbstractLazyLoadRunMap.getById(AbstractLazyLoadRunMap.java:498) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:472) at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1025) at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434) at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189) at hudson.model.Job.getLastFailedBuild(Job.java:823) at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.kohsuke.stapler.export.MethodProperty.getValue(MethodProperty.java:66) at org.kohsuke.stapler.export.Property.writeTo(Property.java:114) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:187) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:182) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:182) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:182) at org.kohsuke.stapler.export.Property.writeValue(Property.java:232) at org.kohsuke.stapler.export.Property.writeValue(Property.java:182) at org.kohsuke.stapler.export.Property.writeValue(Property.java:137) at org.kohsuke.stapler.export.Property.writeTo(Property.java:114) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:187) at org.kohsuke.stapler.export.Model.writeNestedObjectTo(Model.java:182) at org.kohsuke.stapler.export.Model.writeTo(Model.java:154) at hudson.model.Api.doXml(Api.java:103) at sun.reflect.GeneratedMethodAccessor391.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:288) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:151) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:90) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:203) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487) at org.kohsuke.stapler.Stapler.service(Stapler.java:164) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:91) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:864) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1665) at java.lang.Thread.run(Unknown Source) "Handling GET /jenkins/ : http-8080-6": at hudson.model.Actionable.getActions(Actionable.java:73) - waiting to lock <0x00000007871608c8> (a hudson.maven.MavenBuild) at hudson.model.Actionable.getAction(Actionable.java:117) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:351) at hudson.model.Run.onLoad(Run.java:303) at hudson.model.RunMap.retrieve(RunMap.java:221) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638) - locked <0x0000000780a10130> (a hudson.model.RunMap) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621) at jenkins.model.lazy.AbstractLazyLoadRunMap.getById(AbstractLazyLoadRunMap.java:498) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:472) at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1025) at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434) at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189) at hudson.model.Job.getLastFailedBuild(Job.java:823) at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125) at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314) at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185) at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75) at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83) at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57) at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51) at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80) at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72) at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.kohsuke.stapler.jelly.groovy.JellyBuilder.doInvokeMethod(JellyBuilder.java:283) at org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl.invoke(Namespace.java:92) at $Proxy31.projectView(Unknown Source) at lib.JenkinsTagLib$projectView.call(Unknown Source) at hudson.model.View.main.run(main.groovy:14) at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:66) at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:59) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107) at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:562) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:624) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487) at org.kohsuke.stapler.Stapler.service(Stapler.java:164) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:91) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:864) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1665) at java.lang.Thread.run(Unknown Source) "Executor #0 for master : executing DSODP_branch_DEV-1.3 #144": at jenkins.model.lazy.AbstractLazyLoadRunMap.removeValue(AbstractLazyLoadRunMap.java:686) - waiting to lock <0x0000000780a10130> (a hudson.model.RunMap) at hudson.model.RunMap.removeValue(RunMap.java:124) at hudson.model.RunMap.remove(RunMap.java:89) at hudson.model.AbstractProject.removeRun(AbstractProject.java:980) at hudson.model.AbstractProject.removeRun(AbstractProject.java:139) at hudson.model.Run.removeRunFromParent(Run.java:1311) at hudson.model.Run.delete(Run.java:1306) - locked <0x00000007871608c8> (a hudson.maven.MavenBuild) at hudson.model.AbstractBuild.delete(AbstractBuild.java:471) - locked <0x00000007871608c8> (a hudson.maven.MavenBuild) at hudson.maven.MavenModuleSetBuild.delete(MavenModuleSetBuild.java:397) - locked <0x0000000786dc0ed0> (a hudson.maven.MavenModuleSetBuild) at hudson.tasks.LogRotator.perform(LogRotator.java:157) at hudson.model.Job.logRotate(Job.java:381) at hudson.maven.MavenModuleSet.logRotate(MavenModuleSet.java:752) at hudson.model.Run.execute(Run.java:1581) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236)

Andrew Bayer added a comment - 2012-11-27 19:17 This isn't specific to dashboard-view - JENKINS-15929 is the same problem with the jclouds plugin. I've bisected the introduction of the error to 1.489, and am assigning this to kohsuke.

Jesse Glick added a comment - 2013-05-23 15:41 JENKINS-15929 looks unrelated to me.

Jesse Glick added a comment - 2013-05-23 15:53 The two complete thread dumps (gists) do not indicate a deadlock, but rather a performance issue with a MavenBuild having been evicted from memory (or never loaded to begin with) now being loaded, and another thread waiting for that. Such issues might be solved by either deleting excessive historical build records; or running with a large enough heap that more build records can be loaded at once; or changing the Dashboard View plugin to avoid looking up old records: at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:472) at hudson.model.AbstractProject.getNearestOldBuild(AbstractProject.java:1025) at hudson.maven.MavenModuleSetBuild.getModuleLastBuilds(MavenModuleSetBuild.java:434) at hudson.maven.MavenModuleSetBuild.getResult(MavenModuleSetBuild.java:189) at hudson.model.Run.getIconColor(Run.java:640) at hudson.plugins.view.dashboard.stats.StatBuilds.getBuildStat(StatBuilds.java:50) Here StatBuilds is doing something very bad. @edeppe your snippet (please attach a complete thread dump in the future) looks to be unrelated—a true deadlock. Please file it separately.

SCM/JIRA link daemon added a comment - 2013-05-23 19:08 Code changed in jenkins User: Jesse Glick Path: test/src/main/java/org/jvnet/hudson/test/RunLoadCounter.java http://jenkins-ci.org/commit/jenkins/03ca63c62c1f3e20f405b655bbc610ed21931a86 Log: Lazy-loading utility test class designed for fix of JENKINS-15858 . Compare: https://github.com/jenkinsci/jenkins/compare/1551ad6e4683...03ca63c62c1f

Arnaud Héritier added a comment - 2013-05-23 21:43 @jglick I had a look at your PR and it seems good. I don't know if @vmassol can do it otherwise I'll try to test it on my instance this WE

Jesse Glick added a comment - 2013-05-24 00:20 @aheritier field testing would be great! My fix is based on a test only.

Vincent Massol added a comment - 2013-06-25 18:12 And 2 more thread dumps when UI views are slow to load: jstack-vmassol-20130625-2.txt jstack-vmassol-20130625-3.txt

Vincent Massol added a comment - 2013-06-26 06:38 FTR, note that this morning we've had to restart our Jenkins ( http://ci.xwiki.org ) because of OOM: Exception: org.apache.commons.jelly.JellyTagException: jar: file:/home/maven/.hudson/war/WEB-INF/lib/jenkins-core-1.519.jar!/lib/layout/layout.jelly:85:72: <st:include> java.lang.OutOfMemoryError: PermGen space

Kohsuke Kawaguchi added a comment - 2013-07-02 23:07 PermGen space out of space should be tracked separately. If you think you've given sufficient perm gen heap size and Jenkins is overusing permgen space, please follow this guide and obtain the heap dump and send it to us offline.

Kohsuke Kawaguchi added a comment - 2013-07-02 23:19 I looked at the additional thread dumps vmassol-20130625*.txt and I think Jesse's earlier analysis still applies — they are showing that while the UI thread is stuck, it's busy loading records from the disk. I see that three thread dumps involve two jobs "xwiki-platform Quality Checks" and "xwiki-platform". They are both Maven projects with lots of modules, so I can imagine that if the cache is cold, this can result in a considerable delay. When I access this instance, I experience that the first page load time of those two jobs are considerable, yet if I reload the page it renders quickly enough. Between these and the perm gen problem, I suspect that JVM in question simplify doesn't have enough heap size to keep the cache warm enough. Again, I'd love to see the heap dump that I requested above to see if there's something wasting the heap. Another thought that occurred to me is if it helps to provide an option to make the build records a strong reference, instead of the weak reference. It shifts the memory pressure from build records to other soft references, but for some users it might be an useful trade off.

Kohsuke Kawaguchi added a comment - 2013-07-02 23:20 Updated the title of the ticket to reflect the status. There's no dead lock involved.

Jesse Glick added a comment - 2013-07-08 15:20 @vmassol you can try https://buildhive.cloudbees.com/job/jenkinsci/job/dashboard-view-plugin/18/org.jenkins-ci.plugins$dashboard-view/artifact/org.jenkins-ci.plugins/dashboard-view/2.7-SNAPSHOT/dashboard-view-2.7-SNAPSHOT.hpi if you want to still use the Dashboard View plugin.

sogabe added a comment - 2013-07-09 11:17 fixed in 2.8

SCM/JIRA link daemon added a comment - 2016-01-15 19:42 Code changed in jenkins User: Jesse Glick Path: test/src/main/java/org/jvnet/hudson/test/RunLoadCounter.java http://jenkins-ci.org/commit/jenkins-test-harness/9dc74d4132ad4e40874c638dc3f1773c770d87e0 Log: Lazy-loading utility test class designed for fix of JENKINS-15858 . Originally-Committed-As: 03ca63c62c1f3e20f405b655bbc610ed21931a86