Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44564

Deadlock restarting jenkins

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      Jenkins 2.163

      Description

      I'm getting errors uploading plugins to Jenkins and then restarting. This is pretty reliable but doesn't happen on every restart, I have seen it happen about 5 times in the last week, restarting about 20 times a day.

      I was able to kill -3 and get a thread-dump (full text below) – the relevant part seems to be a deadlock described here:

       

      Found one Java-level deadlock:
      =============================
      "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#35]":
      waiting to lock monitor 0x00007f104c0047f8 (object 0x0000000091715c10, a hudson.DescriptorExtensionList),
      which is held by "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#11]"
      "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#11]":
      waiting to lock monitor 0x00007f1070c0d018 (object 0x0000000099f9b518, a hudson.ExtensionList$Lock),
      which is held by "Initializing plugin copyartifact"
      "Initializing plugin copyartifact":
      waiting to lock monitor 0x00007f104c0047f8 (object 0x0000000091715c10, a hudson.DescriptorExtensionList),
      which is held by "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#11]"

      Our work around is to stop the service sudo service jenkins stop - the process will still be running so we kill -9 it, and then start jenkins again.

      Full thread dump – cannot include inline because it exceeds text limits:

      https://gist.github.com/dalvizu/02071f3c8baf30c71cb3df7d2a53ab64

       

        Attachments

          Issue Links

            Activity

            dalvizu Dan Alvizu created issue -
            oleg_nenashev Oleg Nenashev made changes -
            Field Original Value New Value
            Component/s jobconfighistory-plugin [ 15683 ]
            oleg_nenashev Oleg Nenashev made changes -
            Labels deadlock
            oleg_nenashev Oleg Nenashev made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            oleg_nenashev Oleg Nenashev made changes -
            Assignee Stefan Brausch [ stefanbrausch ]
            greybird Arnaud TAMAILLON made changes -
            Link This issue is related to JENKINS-55361 [ JENKINS-55361 ]
            oleg_nenashev Oleg Nenashev made changes -
            Released As Jenkins 2.163
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Fixed but Unreleased [ 10203 ]

              People

              Assignee:
              stefanbrausch Stefan Brausch
              Reporter:
              dalvizu Dan Alvizu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: