• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • core
    • RedHat 5.6 java 1.6. jenkins 1.522

      While parallel deletion/renaming of jobs we have following deadlock:

      Found one Java-level deadlock:
      =============================
      "Handling POST /jenkins/view/QA/view/Verification/createItem : http-bio-8080-exec-189":
        waiting to lock monitor 0x000000004eba7ca8 (object 0x000000070032c4c8, a hudson.model.Hudson),
        which is held by "Handling POST /jenkins/view/RE/view/Onsite/view/Dev/job/RE-Onsite-Build-ATG-Stg/doRename : http-bio-8080-exec-140"
      "Handling POST /jenkins/view/RE/view/Onsite/view/Dev/job/RE-Onsite-Build-ATG-Stg/doRename : http-bio-8080-exec-140":
        waiting to lock monitor 0x000000004f70e1e0 (object 0x0000000752e08ea0, a hudson.maven.MavenModuleSet),
        which is held by "Handling POST /jenkins/view/Maintenance/job/__tst1__/doDelete : http-bio-8080-exec-166"
      "Handling POST /jenkins/view/Maintenance/job/__tst1__/doDelete : http-bio-8080-exec-166":
        waiting to lock monitor 0x000000004eba7ca8 (object 0x000000070032c4c8, a hudson.model.Hudson),
        which is held by "Handling POST /jenkins/view/RE/view/Onsite/view/Dev/job/RE-Onsite-Build-ATG-Stg/doRename : http-bio-8080-exec-140"
      

      Dump of threads:

      "Handling POST /jenkins/view/QA/view/Verification/createItem : http-bio-8080-exec-189":
              at jenkins.model.Jenkins.doCreateItem(Jenkins.java:2889)
              - waiting to lock <0x000000070032c4c8> (a hudson.model.Hudson)
              at jenkins.model.Jenkins.doCreateItem(Jenkins.java:312)
              at hudson.model.ListView.doCreateItem(ListView.java:262)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:297)
              at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:160)
              at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:95)
              at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:586)
              at org.kohsuke.stapler.Stapler.service(Stapler.java:217)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
              at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96)
              at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:208)
              at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:181)
              at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
              at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      
      "Handling POST /jenkins/view/RE/view/Onsite/view/Dev/job/RE-Onsite-Build-ATG-Stg/doRename : http-bio-8080-exec-140":
              at hudson.model.AbstractProject.save(AbstractProject.java:278)
              - waiting to lock <0x0000000752e08ea0> (a hudson.maven.MavenModuleSet)
              at hudson.plugins.copyartifact.CopyArtifact$ListenerImpl.onRenamed(CopyArtifact.java:331)
              at hudson.model.AbstractItem.renameTo(AbstractItem.java:294)
              - locked <0x0000000709ca3ef8> (a hudson.model.FreeStyleProject)
              - locked <0x000000070032c4c8> (a hudson.model.Hudson)
              at hudson.model.Job.renameTo(Job.java:575)
              at hudson.model.Job.doDoRename(Job.java:1294)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:297)
              at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:29)
              at org.kohsuke.stapler.Function$InterceptedFunction.invoke(Function.java:398)
              at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:160)
              at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:95)
              at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:586)
              at org.kohsuke.stapler.Stapler.service(Stapler.java:217)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
      
      "Handling POST /jenkins/view/Maintenance/job/__tst1__/doDelete : http-bio-8080-exec-166":
              at jenkins.model.Jenkins.save(Jenkins.java:2635)
              - waiting to lock <0x000000070032c4c8> (a hudson.model.Hudson)
              at jenkins.model.Jenkins.onDeleted(Jenkins.java:2435)
              at jenkins.model.Jenkins.onDeleted(Jenkins.java:312)
              at hudson.model.AbstractItem.invokeOnDeleted(AbstractItem.java:524)
              at hudson.model.AbstractItem.delete(AbstractItem.java:511)
              - locked <0x0000000752e08ea0> (a hudson.maven.MavenModuleSet)
              at hudson.model.Job.delete(Job.java:585)
              - locked <0x0000000752e08ea0> (a hudson.maven.MavenModuleSet)
              at hudson.model.AbstractProject.doDoDelete(AbstractProject.java:1915)
              at sun.reflect.GeneratedMethodAccessor1652.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:297)
              at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:29)
              at org.kohsuke.stapler.Function$InterceptedFunction.invoke(Function.java:398)
              at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:160)
              at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:95)
              at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:239)
              at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
              at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:684)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:777)
              at org.kohsuke.stapler.Stapler.invoke(Stapler.java:586)
      

          [JENKINS-19446] Deadlock while parallel deletion/rename of jobs

          Joe Zeldin created issue -

          Pull request containing fix: https://github.com/jenkinsci/jenkins/pull/932

          Artem Kirillov added a comment - Pull request containing fix: https://github.com/jenkinsci/jenkins/pull/932

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/model/AbstractItem.java
          http://jenkins-ci.org/commit/jenkins/a5755cb3e1f901c98a2263a3ae1851489cb8e47b
          Log:
          [FIXED JENKINS-19446]

          Defer the notification to avoid deadlock.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/model/AbstractItem.java http://jenkins-ci.org/commit/jenkins/a5755cb3e1f901c98a2263a3ae1851489cb8e47b Log: [FIXED JENKINS-19446] Defer the notification to avoid deadlock.
          SCM/JIRA link daemon made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3085
          [FIXED JENKINS-19446] (Revision a5755cb3e1f901c98a2263a3ae1851489cb8e47b)

          Result = SUCCESS
          kohsuke : a5755cb3e1f901c98a2263a3ae1851489cb8e47b
          Files :

          • core/src/main/java/hudson/model/AbstractItem.java
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #3085 [FIXED JENKINS-19446] (Revision a5755cb3e1f901c98a2263a3ae1851489cb8e47b) Result = SUCCESS kohsuke : a5755cb3e1f901c98a2263a3ae1851489cb8e47b Files : core/src/main/java/hudson/model/AbstractItem.java changelog.html
          Nicolas De Loof made changes -
          Labels New: lts-candidate

          Jesse Glick added a comment -

          The current fix is just no good. It delivers events asynchronously, which makes the system unpredictable, and ruins functional tests.

          Better IMHO would be to call just save() asynchronously.

          Jesse Glick added a comment - The current fix is just no good. It delivers events asynchronously, which makes the system unpredictable, and ruins functional tests. Better IMHO would be to call just save() asynchronously.
          Jesse Glick made changes -
          Resolution Original: Fixed [ 1 ]
          Status Original: Resolved [ 5 ] New: Reopened [ 4 ]

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          http://jenkins-ci.org/commit/jenkins/f22cfbf31b733d8eb64ecdfc7444f7e61ad155b2
          Log:
          JENKINS-19446 Amending changelog; this actually went into 1.544, not 1.543.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html http://jenkins-ci.org/commit/jenkins/f22cfbf31b733d8eb64ecdfc7444f7e61ad155b2 Log: JENKINS-19446 Amending changelog; this actually went into 1.544, not 1.543.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3199
          JENKINS-19446 Amending changelog; this actually went into 1.544, not 1.543. (Revision f22cfbf31b733d8eb64ecdfc7444f7e61ad155b2)

          Result = UNSTABLE
          Jesse Glick : f22cfbf31b733d8eb64ecdfc7444f7e61ad155b2
          Files :

          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #3199 JENKINS-19446 Amending changelog; this actually went into 1.544, not 1.543. (Revision f22cfbf31b733d8eb64ecdfc7444f7e61ad155b2) Result = UNSTABLE Jesse Glick : f22cfbf31b733d8eb64ecdfc7444f7e61ad155b2 Files : changelog.html

            Unassigned Unassigned
            joesephz Joe Zeldin
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: