Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63458

Agent deadlock associated with SlaveInitializer, part III

    XMLWordPrintable

Details

    • 2.254

    Description

      Split from JENKINS-63082:

      Found one Java-level deadlock:
      =============================
      "pool-1-thread-17 for channel id=215328":
        waiting to lock monitor 0x00007f2834022ae8 (object 0x000000076ec31200, a hudson.remoting.RemoteClassLoader),
        which is held by "pool-1-thread-7"
      "pool-1-thread-7":
        waiting to lock monitor 0x00007f2814005a28 (object 0x000000076fc8fda8, a hudson.slaves.SlaveComputer$SlaveInitializer$1),
        which is held by "pool-1-thread-17 for channel id=215328"
      
      Java stack information for the threads listed above:
      ===================================================
      "pool-1-thread-17 for channel id=215328":
      	at hudson.slaves.SlaveComputer$SlaveInitializer$1.publish(SlaveComputer.java:1037)
      	- locked <0x000000076fc8fda8> (a hudson.slaves.SlaveComputer$SlaveInitializer$1)
      	at java.util.logging.Logger.log(Logger.java:738)
      	at java.util.logging.Logger.doLog(Logger.java:765)
      	at java.util.logging.Logger.log(Logger.java:788)
      	at java.util.logging.Logger.fine(Logger.java:1516)
      	at hudson.remoting.Channel.send(Channel.java:764)
      	- locked <0x000000076e47f300> (a hudson.remoting.Channel)
      	at ...
      "pool-1-thread-7":
      	at hudson.slaves.SlaveComputer$SlaveInitializer$1.publish(SlaveComputer.java:1036)
      	- waiting to lock <0x000000076fc8fda8> (a hudson.slaves.SlaveComputer$SlaveInitializer$1)
      	at java.util.logging.Logger.log(Logger.java:738)
      	at java.util.logging.Logger.doLog(Logger.java:765)
      	at java.util.logging.Logger.log(Logger.java:831)
      	at hudson.remoting.RemoteClassLoader.prefetchClassReference(RemoteClassLoader.java:372)
      	at hudson.remoting.RemoteClassLoader.loadWithMultiClassLoader(RemoteClassLoader.java:214)
      	at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:184)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      	- locked <0x000000076ec31200> (a hudson.remoting.RemoteClassLoader)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	at ...
      

      Attachments

        Issue Links

          Activity

            jglick Jesse Glick created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Labels deadlock logging lts-candidate remoting
            jglick Jesse Glick made changes -
            Labels deadlock logging lts-candidate remoting deadlock logging lts-candidate regression remoting
            jglick Jesse Glick made changes -
            Remote Link This issue links to "jenkins #4912 (Web Link)" [ 25700 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-63082 [ JENKINS-63082 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-62181 [ JENKINS-62181 ]
            jglick Jesse Glick added a comment - Build for testing: https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/main/jenkins-war/2.254-rc30357.e908f19c8926/jenkins-war-2.254-rc30357.e908f19c8926.war

            jglick Thanks, that was fast! We'll test it and let you know how it works.

            f5k Fredrik de Vibe added a comment - jglick Thanks, that was fast! We'll test it and let you know how it works.

            jglick, it seems to work great. After upgrading, all the nodes immediately reconnected and everything looks fine. Thank you for an awesome response time!

            f5k Fredrik de Vibe added a comment - jglick , it seems to work great. After upgrading, all the nodes immediately reconnected and everything looks fine. Thank you for an awesome response time!
            timja Tim Jacomb made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            burtsevyg Yuriy Burtsev made changes -
            Comment [ Nodes is up but jobs not running. ]
            jglick Jesse Glick made changes -
            Released As 2.254
            olivergondza Oliver Gondža made changes -
            Labels deadlock logging lts-candidate regression remoting 2.249.1-fixed deadlock logging regression remoting
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-68417 [ JENKINS-68417 ]

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: