-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
None
-
Jenkins 2.66, stash-pullrequest-builder-plugin 1.7.1, openjdk8u131
We are noticing that our Jenkins jvm hangs abruptly and it was impossible to get any thread dumps also. I just got lucky to get the threaddumps today and was able to identify stash-pullrequest-builder-plugin as the root cause.
Our jenkins is heavily used by development teams and having this downtime disrupts our Pipeline Continuous integration.
Here is the thread dump:
"Thread-5682" #13574 daemon prio=5 os_prio=0 tid=0x00007fb88c0e5800 nid=0x263b runnable [0x00007fb834c7f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) - locked < 0x000000068be08ef0> (a java.lang.Object) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940) at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) - locked < 0x000000068be097f8> (a sun.security.ssl.AppInputStream) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) - locked < 0x000000068be09810> (a java.io.BufferedInputStream) at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78) at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106) at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116) at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973) at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient$1.call(StashApiClient.java:230) at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient$1.call(StashApiClient.java:221) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) "jenkins.util.Timer [#6]" #65 daemon prio=5 os_prio=0 tid=0x00007fb850082000 nid=0x266f waiting for monitor entry [0x00007fb8417fb000] java.lang.Thread.State: BLOCKED (on object monitor) at sun.security.ssl.SSLSocketImpl.disposeCiphers(SSLSocketImpl.java:1802) - waiting to lock <0x000000068be08ef0> (a java.lang.Object) at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1738) at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1612) at sun.security.ssl.AppOutputStream.close(AppOutputStream.java:146) at java.io.FilterOutputStream.close(FilterOutputStream.java:159) at org.apache.commons.httpclient.HttpConnection.closeSocketAndStreams(HttpConnection.java:1226) at org.apache.commons.httpclient.HttpConnection.close(HttpConnection.java:1149) at org.apache.commons.httpclient.HttpMethodBase.abort(HttpMethodBase.java:1117) at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getRequest(StashApiClient.java:258) at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getPullRequestComments(StashApiClient.java:109) at stashpullrequestbuilder.stashpullrequestbuilder.StashRepository.isBuildTarget(StashRepository.java:280) at stashpullrequestbuilder.stashpullrequestbuilder.StashRepository.getTargetPullRequests(StashRepository.java:75) at stashpullrequestbuilder.stashpullrequestbuilder.StashPullRequestsBuilder.run(StashPullRequestsBuilder.java:32) at stashpullrequestbuilder.stashpullrequestbuilder.StashBuildTrigger.run(StashBuildTrigger.java:299) at hudson.triggers.Trigger.checkTriggers(Trigger.java:278) at hudson.triggers.Trigger$Cron.doRun(Trigger.java:226) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
