Details
-
Type:
Bug
-
Status: Fixed but Unreleased (View Workflow)
-
Priority:
Major
-
Resolution: Fixed
-
Component/s: core, jobconfighistory-plugin
-
Labels:
-
Similar Issues:
-
Released As:Jenkins 2.163
Description
I'm getting errors uploading plugins to Jenkins and then restarting. This is pretty reliable but doesn't happen on every restart, I have seen it happen about 5 times in the last week, restarting about 20 times a day.
I was able to kill -3 and get a thread-dump (full text below) – the relevant part seems to be a deadlock described here:
Found one Java-level deadlock: ============================= "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#35]": waiting to lock monitor 0x00007f104c0047f8 (object 0x0000000091715c10, a hudson.DescriptorExtensionList), which is held by "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#11]" "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#11]": waiting to lock monitor 0x00007f1070c0d018 (object 0x0000000099f9b518, a hudson.ExtensionList$Lock), which is held by "Initializing plugin copyartifact" "Initializing plugin copyartifact": waiting to lock monitor 0x00007f104c0047f8 (object 0x0000000091715c10, a hudson.DescriptorExtensionList), which is held by "Handling GET /static/5adaeac1/images/headshot.png from 172.30.0.86 : RequestHandlerThread[#11]"
Our work around is to stop the service sudo service jenkins stop - the process will still be running so we kill -9 it, and then start jenkins again.
Full thread dump – cannot include inline because it exceeds text limits:
https://gist.github.com/dalvizu/02071f3c8baf30c71cb3df7d2a53ab64
Attachments
Issue Links
- is related to
-
JENKINS-55361 DescriptorExtensionList not locking correctly, leading to deadlocks
-
- Resolved
-
Activity
Field | Original Value | New Value |
---|---|---|
Component/s | jobconfighistory-plugin [ 15683 ] |
Labels | deadlock |
Priority | Minor [ 4 ] | Major [ 3 ] |
Assignee | Stefan Brausch [ stefanbrausch ] |
Link |
This issue is related to |
Released As | Jenkins 2.163 | |
Resolution | Fixed [ 1 ] | |
Status | Open [ 1 ] | Fixed but Unreleased [ 10203 ] |