Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4220

Deadlock during Maven Build

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Labels:
      None
    • Environment:
      Platform: All, OS: All
    • Similar Issues:

      Description

      Found one Java-level deadlock:
      =============================

      "pool-4-thread-37":
        waiting to lock monitor 0x00007fc070087b48 (object 0x00007fc0849efad8, a java.util.Vector),
        which is held by "RequestHandlerThread[#9]"
      "RequestHandlerThread[#9]":
        waiting to lock monitor 0x00007fc0706fe448 (object 0x00007fc0894de4d8, a hudson.maven.MavenModuleSetBuild),
        which is held by "pool-4-thread-37"
      

      Java stack information for the threads listed above:
      ===================================================

      "pool-4-thread-37":
              at hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:64)
              - waiting to lock <0x00007fc0849efad8> (a java.util.Vector)
              at hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:185)
              at hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:355)
              - locked <0x00007fc0894de4d8> (a hudson.maven.MavenModuleSetBuild)
              at hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:413)
              at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at hudson.model.Executor$1.call(Executor.java:375)
              at hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23)
              at $Proxy11.end(Unknown Source)
              at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:268)
              at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:249)
              at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:209)
              at hudson.remoting.UserRequest.perform(UserRequest.java:103)
              at hudson.remoting.UserRequest.perform(UserRequest.java:47)
              at hudson.remoting.Request$2.run(Request.java:236)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
              at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
              at java.util.concurrent.FutureTask.run(FutureTask.java:138)
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
              at java.lang.Thread.run(Thread.java:619)
      "RequestHandlerThread[#9]":
              at hudson.model.Actionable.getActions(Actionable.java:60)
              - waiting to lock <0x00007fc0894de4d8> (a hudson.maven.MavenModuleSetBuild)
              at hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:208)
              at hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:63)
              at hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:72)
              - locked <0x00007fc0849efad8> (a java.util.Vector)
              at hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:185)
              at hudson.model.AbstractProject.submit(AbstractProject.java:1289)
              at hudson.maven.AbstractMavenProject.submit(AbstractMavenProject.java:136)
              at hudson.maven.MavenModuleSet.submit(MavenModuleSet.java:620)
              at hudson.model.Job.doConfigSubmit(Job.java:932)
              - locked <0x00007fc083f84280> (a hudson.maven.MavenModuleSet)
              at hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:479)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:176)
              at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:92)
              at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:53)
              at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:73)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:486)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:179)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:486)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:404)
              at org.kohsuke.stapler.Stapler.service(Stapler.java:117)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
              at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
              at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
              at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
              at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
              at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
              at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
              at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
              at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
              at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:155)
              at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
              at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
              at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
              at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
              at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
              at java.lang.Thread.run(Thread.java:619)
      
      Found 1 deadlock.
      

        Attachments

          Issue Links

            Activity

            Hide
            evernat evernat added a comment -

            I confirm this deadlock with hudson 1.339.

            The "Monitoring" hudson plugin says in red: "Warning, the following threads are deadlocked : pool-3-thread-2066, RequestHandlerThread14" (based on java.lang.management.ThreadMXBean.findDeadlockedThreads())

            The stack-traces of the 2 threads are:
            Thread pool-3-thread-2066:
            hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:65)
            hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199)
            hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:358)
            hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:419)
            sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
            sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            java.lang.reflect.Method.invoke(Method.java:597)
            hudson.model.Executor$1.call(Executor.java:375)
            hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23)
            $Proxy22.end(Unknown Source)
            sun.reflect.GeneratedMethodAccessor2488.invoke(Unknown Source)
            sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            java.lang.reflect.Method.invoke(Method.java:597)
            hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274)
            hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:255)
            hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:215)
            hudson.remoting.UserRequest.perform(UserRequest.java:104)
            hudson.remoting.UserRequest.perform(UserRequest.java:48)
            hudson.remoting.Request$2.run(Request.java:270)
            java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
            java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
            java.util.concurrent.FutureTask.run(FutureTask.java:138)
            java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
            java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
            java.lang.Thread.run(Thread.java:619)

            Thread RequestHandlerThread14:
            hudson.model.Actionable.getActions(Actionable.java:60)
            hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:222)
            hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:64)
            hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:72)
            hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199)
            hudson.model.AbstractProject.submit(AbstractProject.java:1409)
            hudson.maven.MavenModuleSet.submit(MavenModuleSet.java:694)
            hudson.model.Job.doConfigSubmit(Job.java:967)
            hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:527)
            sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            java.lang.reflect.Method.invoke(Method.java:597)
            org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:185)
            org.kohsuke.stapler.Function.bindAndInvoke(Function.java:101)
            org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:54)
            org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:74)
            org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
            org.kohsuke.stapler.Stapler.invoke(Stapler.java:492)
            org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:180)
            org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
            org.kohsuke.stapler.Stapler.invoke(Stapler.java:492)
            org.kohsuke.stapler.Stapler.invoke(Stapler.java:408)
            org.kohsuke.stapler.Stapler.service(Stapler.java:117)
            javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
            winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
            winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
            winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
            hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
            net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:304)
            org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:31)
            hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
            hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
            winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
            winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
            hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
            winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
            winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
            hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
            hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
            winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
            winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
            winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
            winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
            winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
            java.lang.Thread.run(Thread.java:619)

            Show
            evernat evernat added a comment - I confirm this deadlock with hudson 1.339. The "Monitoring" hudson plugin says in red: "Warning, the following threads are deadlocked : pool-3-thread-2066, RequestHandlerThread 14 " (based on java.lang.management.ThreadMXBean.findDeadlockedThreads()) The stack-traces of the 2 threads are: Thread pool-3-thread-2066: hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:65) hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199) hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:358) hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:419) sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) hudson.model.Executor$1.call(Executor.java:375) hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23) $Proxy22.end(Unknown Source) sun.reflect.GeneratedMethodAccessor2488.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274) hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:255) hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:215) hudson.remoting.UserRequest.perform(UserRequest.java:104) hudson.remoting.UserRequest.perform(UserRequest.java:48) hudson.remoting.Request$2.run(Request.java:270) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) java.util.concurrent.FutureTask.run(FutureTask.java:138) java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) java.lang.Thread.run(Thread.java:619) Thread RequestHandlerThread 14 : hudson.model.Actionable.getActions(Actionable.java:60) hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:222) hudson.maven.MavenModuleSet.addTransientActionsFromBuild(MavenModuleSet.java:64) hudson.maven.AbstractMavenProject.updateTransientActions(AbstractMavenProject.java:72) hudson.maven.MavenModuleSet.updateTransientActions(MavenModuleSet.java:199) hudson.model.AbstractProject.submit(AbstractProject.java:1409) hudson.maven.MavenModuleSet.submit(MavenModuleSet.java:694) hudson.model.Job.doConfigSubmit(Job.java:967) hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:527) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:185) org.kohsuke.stapler.Function.bindAndInvoke(Function.java:101) org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:54) org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:74) org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30) org.kohsuke.stapler.Stapler.invoke(Stapler.java:492) org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:180) org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30) org.kohsuke.stapler.Stapler.invoke(Stapler.java:492) org.kohsuke.stapler.Stapler.invoke(Stapler.java:408) org.kohsuke.stapler.Stapler.service(Stapler.java:117) javax.servlet.http.HttpServlet.service(HttpServlet.java:45) winstone.ServletConfiguration.execute(ServletConfiguration.java:249) winstone.RequestDispatcher.forward(RequestDispatcher.java:335) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94) net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:304) org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:31) hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97) hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86) winstone.FilterConfiguration.execute(FilterConfiguration.java:195) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) winstone.FilterConfiguration.execute(FilterConfiguration.java:195) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) winstone.FilterConfiguration.execute(FilterConfiguration.java:195) winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) winstone.RequestDispatcher.forward(RequestDispatcher.java:333) winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) java.lang.Thread.run(Thread.java:619)
            Hide
            evernat evernat added a comment -

            I think that this issue duplicates issue 7790 which is fixed in v1.381.
            Could we close this one?

            Show
            evernat evernat added a comment - I think that this issue duplicates issue 7790 which is fixed in v1.381. Could we close this one?
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            changelog.html
            maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
            http://jenkins-ci.org/commit/core/53a31c31530166fc7a53900b58127c002a7e8cda
            Log:
            [FIXED JENKINS-4220] fixed a dead lock.

            While the original report is in https://cloudbees.zendesk.com/tickets/427, this fix also takes care of JENKINS-4220.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java http://jenkins-ci.org/commit/core/53a31c31530166fc7a53900b58127c002a7e8cda Log: [FIXED JENKINS-4220] fixed a dead lock. While the original report is in https://cloudbees.zendesk.com/tickets/427 , this fix also takes care of JENKINS-4220 .
            Hide
            dogfood dogfood added a comment -
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #548
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/maven/MavenModuleSetBuild.java
            http://jenkins-ci.org/commit/maven-plugin/a901e3d3711dd368745039c283661e463d11ec65
            Log:
            [FIXED JENKINS-4220] fixed a dead lock.

            While the original report is in https://cloudbees.zendesk.com/tickets/427, this fix also takes care of JENKINS-4220.

            Originally-Committed-As: 53a31c31530166fc7a53900b58127c002a7e8cda

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/maven/MavenModuleSetBuild.java http://jenkins-ci.org/commit/maven-plugin/a901e3d3711dd368745039c283661e463d11ec65 Log: [FIXED JENKINS-4220] fixed a dead lock. While the original report is in https://cloudbees.zendesk.com/tickets/427 , this fix also takes care of JENKINS-4220 . Originally-Committed-As: 53a31c31530166fc7a53900b58127c002a7e8cda

              People

              Assignee:
              cmj Carsten Rohrbach
              Reporter:
              cmj Carsten Rohrbach
              Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: