Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15640

Deadlock in GroovyShell.<init>

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      A user encountered a hung startup after around a minute of activity. Successive thread dumps showed that initialization was paused here:

      "ModelList.init" daemon prio=10 … in Object.wait() …
         java.lang.Thread.State: RUNNABLE
      	at org.codehaus.groovy.util.ReferenceBundle.<clinit>(ReferenceBundle.java:37)
      	at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.<init>(MetaClassRegistryImpl.java:53)
      	at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.<init>(MetaClassRegistryImpl.java:61)
      	at groovy.lang.GroovySystem.<clinit>(GroovySystem.java:29)
      	at org.codehaus.groovy.runtime.InvokerHelper.<clinit>(InvokerHelper.java:49)
      	at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:32)
      	at groovy.lang.Binding.<init>(Binding.java:34)
      	at groovy.lang.GroovyShell.<init>(GroovyShell.java:82)
      	at com.cloudbees.hudson.plugins.modeling.ModelList.init(ModelList.java:71)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:616)
      	at hudson.init.InitializerFinder.invoke(InitializerFinder.java:120)
      	at hudson.init.InitializerFinder$TaskImpl.run(InitializerFinder.java:184)
      	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259)
      	at jenkins.model.Jenkins$6.runTask(Jenkins.java:840)
      	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187)
      	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      	at java.lang.Thread.run(Thread.java:636)
      

      The line of code in question (assuming Groovy 1.8.5):

      ReferenceManager callBack = ReferenceManager.createCallBackedManager(queue);
      

      There were tons of request threads, mostly hung in something like:

      "Handling GET /…some page… : http-8080-2" daemon prio=10 … in Object.wait() …
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00002aaabb3400a8> (a com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference)
      	at java.lang.Object.wait(Object.java:502)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.waitForValue(ComputingConcurrentHashMap.java:328)
      	- locked <0x00002aaabb3400a8> (a com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute(ComputingConcurrentHashMap.java:160)
      	at com.google.common.collect.ComputingConcurrentHashMap.getOrCompute(ComputingConcurrentHashMap.java:69)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393)
      	at org.kohsuke.stapler.CachingScriptLoader.findScript(CachingScriptLoader.java:61)
      	at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:112)
      	…
      

      But one was different:

      "Handling GET / : http-8080-1" daemon prio=10 … in Object.wait() …
         java.lang.Thread.State: RUNNABLE
      	at org.codehaus.groovy.util.ReferenceManager.<clinit>(ReferenceManager.java:153)
      	at org.codehaus.groovy.util.ManagedReference.<clinit>(ManagedReference.java:25)
      	at org.codehaus.groovy.reflection.ReflectionCache.getCachedClass(ReflectionCache.java:107)
      	at org.codehaus.groovy.reflection.ReflectionCache.<clinit>(ReflectionCache.java:52)
      	at org.codehaus.groovy.classgen.asm.BytecodeHelper.box(BytecodeHelper.java:585)
      	at org.codehaus.groovy.classgen.asm.BytecodeHelper.box(BytecodeHelper.java:577)
      	at org.codehaus.groovy.classgen.asm.OperandStack.box(OperandStack.java:183)
      	at org.codehaus.groovy.classgen.asm.InvocationWriter.makeCall(InvocationWriter.java:220)
      	at org.codehaus.groovy.classgen.asm.InvocationWriter.makeInvokeMethodCall(InvocationWriter.java:66)
      	at org.codehaus.groovy.classgen.asm.InvocationWriter.writeInvokeMethod(InvocationWriter.java:292)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitMethodCallExpression(AsmClassGenerator.java:655)
      	at org.codehaus.groovy.ast.expr.MethodCallExpression.visit(MethodCallExpression.java:75)
      	at org.codehaus.groovy.classgen.asm.StatementWriter.writeExpressionStatement(StatementWriter.java:599)
      	at org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeExpressionStatement(OptimizingStatementWriter.java:354)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitExpressionStatement(AsmClassGenerator.java:501)
      	at org.codehaus.groovy.ast.stmt.ExpressionStatement.visit(ExpressionStatement.java:40)
      	at org.codehaus.groovy.classgen.asm.StatementWriter.writeBlockStatement(StatementWriter.java:80)
      	at org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeBlockStatement(OptimizingStatementWriter.java:155)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitBlockStatement(AsmClassGenerator.java:447)
      	at org.codehaus.groovy.ast.stmt.BlockStatement.visit(BlockStatement.java:69)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClassCodeContainer(ClassCodeVisitorSupport.java:101)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructorOrMethod(ClassCodeVisitorSupport.java:112)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitStdMethod(AsmClassGenerator.java:312)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructorOrMethod(AsmClassGenerator.java:269)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructor(ClassCodeVisitorSupport.java:119)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructor(AsmClassGenerator.java:384)
      	at org.codehaus.groovy.ast.ClassNode.visitContents(ClassNode.java:1052)
      	at org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClass(ClassCodeVisitorSupport.java:50)
      	at org.codehaus.groovy.classgen.AsmClassGenerator.visitClass(AsmClassGenerator.java:173)
      	at org.codehaus.groovy.control.CompilationUnit$14.call(CompilationUnit.java:767)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:967)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:546)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:524)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:501)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:306)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:287)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
      	at org.kohsuke.stapler.jelly.groovy.GroovyClassLoaderTearOff.parse(GroovyClassLoaderTearOff.java:82)
      	at org.kohsuke.stapler.jelly.groovy.GroovyClassTearOff.parseScript(GroovyClassTearOff.java:57)
      	at org.kohsuke.stapler.jelly.groovy.GroovyClassTearOff.parseScript(GroovyClassTearOff.java:46)
      	at org.kohsuke.stapler.AbstractTearOff.resolveScript(AbstractTearOff.java:72)
      	at org.kohsuke.stapler.jelly.JellyClassTearOff.resolveScript(JellyClassTearOff.java:80)
      	at org.kohsuke.stapler.jelly.JellyClassTearOff.resolveScript(JellyClassTearOff.java:47)
      	at org.kohsuke.stapler.AbstractTearOff.loadScript(AbstractTearOff.java:78)
      	at org.kohsuke.stapler.CachingScriptLoader$1.apply(CachingScriptLoader.java:30)
      	at org.kohsuke.stapler.CachingScriptLoader$1.apply(CachingScriptLoader.java:27)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingValueReference.compute(ComputingConcurrentHashMap.java:355)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.compute(ComputingConcurrentHashMap.java:184)
      	- locked <0x00002aaabb340170> (a com.google.common.collect.MapMakerInternalMap$StrongEntry)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingSegment.getOrCompute(ComputingConcurrentHashMap.java:153)
      	at com.google.common.collect.ComputingConcurrentHashMap.getOrCompute(ComputingConcurrentHashMap.java:69)
      	at com.google.common.collect.ComputingConcurrentHashMap$ComputingMapAdapter.get(ComputingConcurrentHashMap.java:393)
      	at org.kohsuke.stapler.CachingScriptLoader.findScript(CachingScriptLoader.java:61)
      	at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:112)
      	…
      

      which is in:

      SOFT_BUNDLE = new ReferenceBundle(manager, ReferenceType.SOFT);
      

      Seems like a class loading deadlock: two static initializers referring to one another, and two threads each initially referring to one of these classes.

      Groovy 2.0.5 has the same code. Cannot find any existing reports of this, so it is probably a rare race condition.

        Attachments

          Activity

          There are no comments yet on this issue.

            People

            Assignee:
            jglick Jesse Glick
            Reporter:
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: