Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50663

Deadlock on jenkins startup

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Blocker
    • Resolution: Fixed
    • core, sonar-plugin
    • Jenkins version 2.114
      Plugins versions:
      * sonar: 2.7
      * git 3.8.0
      * branch-api 2.0.19
    • Jenkins 2.163

    Description

      After plugin update jenkins hangs on startup. Attached thread dump showing deadlock. Here's short version:

      Found one Java-level deadlock:
      =============================
      "CustomOrganizationFolderDescriptor.addSpecificDescriptors":
        waiting to lock monitor 0x00007f5510004f38 (object 0x00000006c6654678, a hudson.ExtensionList$Lock),
        which is held by "GitSCM.onLoaded"
      "GitSCM.onLoaded":
        waiting for ownable synchronizer 0x00000006c6eb7488, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
        which is held by "SonarGlobalConfiguration.migrate"
      "SonarGlobalConfiguration.migrate":
        waiting to lock monitor 0x00007f5510004f38 (object 0x00000006c6654678, a hudson.ExtensionList$Lock),
        which is held by "GitSCM.onLoaded"
      

      Attachments

        Issue Links

          Activity

            jglick Jesse Glick added a comment -

            While a Sonar plugin change may have triggered this bug, it looks more like a core issue to me. The deadlock is between GuiceFinder and ExtensionList, but possibly OldDataMonitor.report should not be acquiring a lock either since it is involved at a low level in initialization—perhaps it could queue up reports until after startup has completed, or at least extensions have been prepared, or perhaps data could become a static WeakHashMap<Jenkins, …> reference.

            jglick Jesse Glick added a comment - While a Sonar plugin change may have triggered this bug, it looks more like a core issue to me. The deadlock is between GuiceFinder and ExtensionList , but possibly OldDataMonitor.report should not be acquiring a lock either since it is involved at a low level in initialization—perhaps it could queue up reports until after startup has completed, or at least extensions have been prepared, or perhaps data could become a static WeakHashMap<Jenkins, …> reference.
            owenmehegan Owen Mehegan added a comment -

            henryju please consider submitting a request to remove 2.7.0 from the update center, given that installing it can render Jenkins unable to start at all. See https://jenkins.io/doc/developer/publishing/removing-from-distribution/

            owenmehegan Owen Mehegan added a comment - henryju please consider submitting a request to remove 2.7.0 from the update center, given that installing it can render Jenkins unable to start at all. See  https://jenkins.io/doc/developer/publishing/removing-from-distribution/

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            src/test/java/org/jenkinsci/plugins/pipeline/utility/steps/conf/mf/ReadManifestStepTest.java
            http://jenkins-ci.org/commit/pipeline-utility-steps-plugin/982516b2fc8c2f641fb362da915f6bb9a037d76b
            Log:
            JENKINS-50663 - Restore Remoting manifest checks as requested by @rsandell

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/test/java/org/jenkinsci/plugins/pipeline/utility/steps/conf/mf/ReadManifestStepTest.java http://jenkins-ci.org/commit/pipeline-utility-steps-plugin/982516b2fc8c2f641fb362da915f6bb9a037d76b Log: JENKINS-50663 - Restore Remoting manifest checks as requested by @rsandell
            dlouzado Dario Louzado added a comment -

            In my case, updating to version 2.7.1 did not fix the problem.

            2.7.1 did work on a new installation and did not on my staging environment. Jenkins startup freezed again and no logs were written. Cannot use this version on production env.

            dlouzado Dario Louzado added a comment - In my case, updating to version 2.7.1 did not fix the problem. 2.7.1 did work on a new installation and did not on my staging environment. Jenkins startup freezed again and no logs were written. Cannot use this version on production env.
            oleg_nenashev Oleg Nenashev added a comment -

             

            JENKINS-55361 should fix it in 2.163 from what I see

            oleg_nenashev Oleg Nenashev added a comment -   JENKINS-55361 should fix it in 2.163 from what I see

            People

              Unassigned Unassigned
              jchorko Janusz Chorko
              Votes:
              5 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: