For more than a week, we have experienced deadlocks every morning.
We suspect it occurs when someone hits the web ui for the first time that day. We do have nightly rebuilds, but the deadlock doesn't appear to occur until much later in the morning. The deadlocks don't occur at the same time, but they are consistent every morning.
We have tried a couple things so far:
Upgrade from 1.520 to 1.525
Replace Winstone with Jetty (suggested by nanoyme #jenkins@freenode)
I've attached the jstack traces for when jenkins gets wedged and after a fresh restart as well as the plugins list.
Would like some pointers on what to diagnose next. We were hoping replacing winstone with jetty was going to work.