• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None
    • Platform: All, OS: All

      Found one Java-level deadlock:
      =============================

      "pool-4-thread-37":
        waiting to lock monitor 0x00007fc070087b48 (object 0x00007fc0849efad8, a java.util.Vector),
        which is held by "RequestHandlerThread[#9]"
      "RequestHandlerThread[#9]":
        waiting to lock monitor 0x00007fc0706fe448 (object 0x00007fc0894de4d8, a hudson.maven.MavenModuleSetBuild),
        which is held by "pool-4-thread-37"
      

      Java stack information for the threads listed above:
      ===================================================

      "pool-4-thread-37":
              at hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:64)
              - waiting to lock <0x00007fc0849efad8> (a java.util.Vector)
              at hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:185)
              at hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:355)
              - locked <0x00007fc0894de4d8> (a hudson.maven.MavenModuleSetBuild)
              at hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:413)
              at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at hudson.model.Executor$1.call(Executor.java:375)
              at hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23)
              at $Proxy11.end(Unknown Source)
              at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:268)
              at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:249)
              at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:209)
              at hudson.remoting.UserRequest.perform(UserRequest.java:103)
              at hudson.remoting.UserRequest.perform(UserRequest.java:47)
              at hudson.remoting.Request$2.run(Request.java:236)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
              at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
              at java.util.concurrent.FutureTask.run(FutureTask.java:138)
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
              at java.lang.Thread.run(Thread.java:619)
      "RequestHandlerThread[#9]":
              at hudson.model.Actionable.getActions(Actionable.java:60)
              - waiting to lock <0x00007fc0894de4d8> (a hudson.maven.MavenModuleSetBuild)
              at hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:208)
              at hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:63)
              at hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:72)
              - locked <0x00007fc0849efad8> (a java.util.Vector)
              at hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:185)
              at hudson.model.AbstractProject.submit(AbstractProject.java:1289)
              at hudson.maven.AbstractMavenProject.submit(AbstractMavenProject.java:136)
              at hudson.maven.MavenModuleSet.submit(MavenModuleSet.java:620)
              at hudson.model.Job.doConfigSubmit(Job.java:932)
              - locked <0x00007fc083f84280> (a hudson.maven.MavenModuleSet)
              at hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:479)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:176)
              at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:92)
              at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:53)
              at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:73)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:486)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:179)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:486)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:404)
              at org.kohsuke.stapler.Stapler.service(Stapler.java:117)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
              at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
              at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
              at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
              at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
              at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
              at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
              at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
              at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:155)
              at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
              at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
              at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
              at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
              at java.lang.Thread.run(Thread.java:619)
      
      Found 1 deadlock.
      

          [JENKINS-4220] Deadlock during Maven Build

          evernat added a comment -

          I confirm this deadlock with hudson 1.339.

          The "Monitoring" hudson plugin says in red: "Warning, the following threads are deadlocked : pool-3-thread-2066, RequestHandlerThread14" (based on java.lang.management.ThreadMXBean.findDeadlockedThreads())

          The stack-traces of the 2 threads are:
          Thread pool-3-thread-2066:
          hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:65)
          hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199)
          hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:358)
          hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:419)
          sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          java.lang.reflect.Method.invoke(Method.java:597)
          hudson.model.Executor$1.call(Executor.java:375)
          hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23)
          $Proxy22.end(Unknown Source)
          sun.reflect.GeneratedMethodAccessor2488.invoke(Unknown Source)
          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          java.lang.reflect.Method.invoke(Method.java:597)
          hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274)
          hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:255)
          hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:215)
          hudson.remoting.UserRequest.perform(UserRequest.java:104)
          hudson.remoting.UserRequest.perform(UserRequest.java:48)
          hudson.remoting.Request$2.run(Request.java:270)
          java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
          java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          java.util.concurrent.FutureTask.run(FutureTask.java:138)
          java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          java.lang.Thread.run(Thread.java:619)

          Thread RequestHandlerThread14:
          hudson.model.Actionable.getActions(Actionable.java:60)
          hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:222)
          hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:64)
          hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:72)
          hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199)
          hudson.model.AbstractProject.submit(AbstractProject.java:1409)
          hudson.maven.MavenModuleSet.submit(MavenModuleSet.java:694)
          hudson.model.Job.doConfigSubmit(Job.java:967)
          hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:527)
          sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          java.lang.reflect.Method.invoke(Method.java:597)
          org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:185)
          org.kohsuke.stapler.Function.bindAndInvoke(Function.java:101)
          org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:54)
          org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:74)
          org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
          org.kohsuke.stapler.Stapler.invoke(Stapler.java:492)
          org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:180)
          org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
          org.kohsuke.stapler.Stapler.invoke(Stapler.java:492)
          org.kohsuke.stapler.Stapler.invoke(Stapler.java:408)
          org.kohsuke.stapler.Stapler.service(Stapler.java:117)
          javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
          winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
          winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
          winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
          hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
          net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:304)
          org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:31)
          hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
          hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
          winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
          winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
          hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
          winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
          winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
          hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
          hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
          hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
          winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
          winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
          winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
          winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
          winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
          java.lang.Thread.run(Thread.java:619)

          evernat added a comment - I confirm this deadlock with hudson 1.339. The "Monitoring" hudson plugin says in red: "Warning, the following threads are deadlocked : pool-3-thread-2066, RequestHandlerThread 14 " (based on java.lang.management.ThreadMXBean.findDeadlockedThreads()) The stack-traces of the 2 threads are: Thread pool-3-thread-2066: hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:65) hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199) hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:358) hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:419) sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) hudson.model.Executor$1.call(Executor.java:375) hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23) $Proxy22.end(Unknown Source) sun.reflect.GeneratedMethodAccessor2488.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274) hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:255) hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:215) hudson.remoting.UserRequest.perform(UserRequest.java:104) hudson.remoting.UserRequest.perform(UserRequest.java:48) hudson.remoting.Request$2.run(Request.java:270) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) java.util.concurrent.FutureTask.run(FutureTask.java:138) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) java.lang.Thread.run(Thread.java:619) Thread RequestHandlerThread 14 : hudson.model.Actionable.getActions(Actionable.java:60) hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:222) hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:64) hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:72) hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199) hudson.model.AbstractProject.submit(AbstractProject.java:1409) hudson.maven.MavenModuleSet.submit(MavenModuleSet.java:694) hudson.model.Job.doConfigSubmit(Job.java:967) hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:527) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:185) org.kohsuke.stapler.Function.bindAndInvoke(Function.java:101) org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:54) org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:74) org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30) org.kohsuke.stapler.Stapler.invoke(Stapler.java:492) org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:180) org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30) org.kohsuke.stapler.Stapler.invoke(Stapler.java:492) org.kohsuke.stapler.Stapler.invoke(Stapler.java:408) org.kohsuke.stapler.Stapler.service(Stapler.java:117) javax.servlet.http.HttpServlet.service(HttpServlet.java:45) winstone.ServletConfiguration.execute(ServletConfiguration.java:249) winstone.RequestDispatcher.forward(RequestDispatcher.java:335) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94) net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:304) org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:31) hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97) hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86) winstone.FilterConfiguration.execute(FilterConfiguration.java:195) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) winstone.FilterConfiguration.execute(FilterConfiguration.java:195) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) winstone.FilterConfiguration.execute(FilterConfiguration.java:195) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) winstone.RequestDispatcher.forward(RequestDispatcher.java:333) winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) java.lang.Thread.run(Thread.java:619)

          evernat added a comment -

          I think that this issue duplicates issue 7790 which is fixed in v1.381.
          Could we close this one?

          evernat added a comment - I think that this issue duplicates issue 7790 which is fixed in v1.381. Could we close this one?

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          http://jenkins-ci.org/commit/core/53a31c31530166fc7a53900b58127c002a7e8cda
          Log:
          [FIXED JENKINS-4220] fixed a dead lock.

          While the original report is in https://cloudbees.zendesk.com/tickets/427, this fix also takes care of JENKINS-4220.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java http://jenkins-ci.org/commit/core/53a31c31530166fc7a53900b58127c002a7e8cda Log: [FIXED JENKINS-4220] fixed a dead lock. While the original report is in https://cloudbees.zendesk.com/tickets/427 , this fix also takes care of JENKINS-4220 .

          dogfood added a comment -

          dogfood added a comment - Integrated in jenkins_main_trunk #548

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          src/main/java/hudson/maven/MavenModuleSetBuild.java
          http://jenkins-ci.org/commit/maven-plugin/a901e3d3711dd368745039c283661e463d11ec65
          Log:
          [FIXED JENKINS-4220] fixed a dead lock.

          While the original report is in https://cloudbees.zendesk.com/tickets/427, this fix also takes care of JENKINS-4220.

          Originally-Committed-As: 53a31c31530166fc7a53900b58127c002a7e8cda

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/maven/MavenModuleSetBuild.java http://jenkins-ci.org/commit/maven-plugin/a901e3d3711dd368745039c283661e463d11ec65 Log: [FIXED JENKINS-4220] fixed a dead lock. While the original report is in https://cloudbees.zendesk.com/tickets/427 , this fix also takes care of JENKINS-4220 . Originally-Committed-As: 53a31c31530166fc7a53900b58127c002a7e8cda

            cmj Carsten Rohrbach
            cmj Carsten Rohrbach
            Votes:
            5 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: