Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43197

Deadlock when many jobs running

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Cannot Reproduce
    • Component/s: core
    • Labels:
      None
    • Environment:
      Jenkins 2.48
      xUnit plugin 1.102
      JUnit Plugin 1.20
      Plot Plugin 1.9
      RHEL6.3, Java 1.8.0_92
    • Similar Issues:

      Description

      After update of Jenkins from 2.8 to 2.48 nodes get stuck from time to time.
      Thread dump shows:

      "Executor #0 for my_node : executing my_job #510" #266539 daemon prio=5 os_prio=0 tid=0x00007f5fc1268800 nid=0x4885 waiting for monitor entry [0x00007f5f471f3000]
         java.lang.Thread.State: BLOCKED (on object monitor)
          at org.codehaus.groovy.reflection.ClassInfo$GlobalClassSet.add(ClassInfo.java:477)
          - waiting to lock <0x00007f621754d9c0> (a org.codehaus.groovy.util.ManagedLinkedList)
          at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:83)
          at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:79)
          at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.<init>(GroovyClassValuePreJava7.java:37)
          at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:64)
          at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:55)
          at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.put(AbstractConcurrentMap.java:157)
          at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.getOrPut(AbstractConcurrentMap.java:100)
          at org.codehaus.groovy.util.AbstractConcurrentMap.getOrPut(AbstractConcurrentMap.java:38)
          at org.codehaus.groovy.reflection.GroovyClassValuePreJava7.get(GroovyClassValuePreJava7.java:94)
          at org.codehaus.groovy.reflection.ClassInfo.getClassInfo(ClassInfo.java:144)
          at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.getMetaClass(MetaClassRegistryImpl.java:258)
          at org.codehaus.groovy.runtime.InvokerHelper.getMetaClass(InvokerHelper.java:883)
          at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:34)
          at groovy.lang.Script.<init>(Script.java:42)
      :
      

      In changelog we found that with Jenkins version 2.47 Groovy was updated to 2.4.8 "to address memory leak issue".

      Possibly this issue is related to

      https://issues.apache.org/jira/browse/GROOVY-8067: Possible deadlock when creating new ClassInfo entries in the cache

      which is in Groovy 2.4.8 and solved in 2.4.9

       

       

        Attachments

          Issue Links

            Activity

            hjhafner Hans-Juergen Hafner created issue -
            hjhafner Hans-Juergen Hafner made changes -
            Field Original Value New Value
            Description After update of Jenkins from 2.8 to 2.48 nodes get stuck from time to time.
            Thread dump shows:
            {code:java}
            "Executor #0 for my_node : executing my_job #510" #266539 daemon prio=5 os_prio=0 tid=0x00007f5fc1268800 nid=0x4885 waiting for monitor entry [0x00007f5f471f3000]
               java.lang.Thread.State: BLOCKED (on object monitor)
                at org.codehaus.groovy.reflection.ClassInfo$GlobalClassSet.add(ClassInfo.java:477)
                - waiting to lock <0x00007f621754d9c0> (a org.codehaus.groovy.util.ManagedLinkedList)
                at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:83)
                at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:79)
                at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.<init>(GroovyClassValuePreJava7.java:37)
                at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:64)
                at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:55)
                at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.put(AbstractConcurrentMap.java:157)
                at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.getOrPut(AbstractConcurrentMap.java:100)
                at org.codehaus.groovy.util.AbstractConcurrentMap.getOrPut(AbstractConcurrentMap.java:38)
                at org.codehaus.groovy.reflection.GroovyClassValuePreJava7.get(GroovyClassValuePreJava7.java:94)
                at org.codehaus.groovy.reflection.ClassInfo.getClassInfo(ClassInfo.java:144)
                at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.getMetaClass(MetaClassRegistryImpl.java:258)
                at org.codehaus.groovy.runtime.InvokerHelper.getMetaClass(InvokerHelper.java:883)
                at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:34)
                at groovy.lang.Script.<init>(Script.java:42)
            :
            {code}
            In changelog we found that with Jenkins version 2.47 Groovy was updated to 2.4.8 "_to address memory leak issue_".

            Possibly this issue is related to

            [https://issues.apache.org/jira/browse/GROOVY-8067: |https://issues.apache.org/jira/browse/GROOVY-8067]*Possible deadlock when creating new ClassInfo entries in the cache*

            which is in Groovy 2.4.8 and solved in 2.4.9

             

             
            io39d io39d made changes -
            Link This issue relates to JENKINS-42637 [ JENKINS-42637 ]
            danielbeck Daniel Beck made changes -
            Resolution Cannot Reproduce [ 5 ]
            Status Open [ 1 ] Resolved [ 5 ]
            svanoort Sam Van Oort made changes -
            Link This issue is duplicated by JENKINS-46709 [ JENKINS-46709 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              hjhafner Hans-Juergen Hafner
              Votes:
              6 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: