Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41693

Stuck socket communication can block aync disposer

XMLWordPrintable

      Foreman resource was found in disposer queue to over a day with not progress. In fact, the disposable thread got stuck in formean code:

      "Computer.threadPoolForRemoting [#6845]" #287774 daemon prio=5
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
      	at java.net.SocketInputStream.read(SocketInputStream.java:170)
      	at java.net.SocketInputStream.read(SocketInputStream.java:141)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
      	- locked <0x686ff29c> (a java.io.BufferedInputStream)
      	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
      	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
      	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
      	- locked <0x7bfced60> (a sun.net.www.protocol.http.HttpURLConnection)
      	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
      	- locked <0x7bfced60> (a sun.net.www.protocol.http.HttpURLConnection)
      	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
      	at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:394)
      	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285)
      	at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:255)
      	at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
      	at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
      	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
      	at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
      	at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411)
      	at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:311)
      	at com.redhat.foreman.ForemanAPI.getForemanResponse(ForemanAPI.java:185)
      	at com.redhat.foreman.ForemanAPI.getHostParameterValue(ForemanAPI.java:231)
      	at com.redhat.foreman.ForemanAPI.isHostFree(ForemanAPI.java:393)
      	at com.redhat.foreman.ForemanAPI.release(ForemanAPI.java:149)
      	at com.redhat.foreman.DisposableImpl.dispose(DisposableImpl.java:30)
      	at org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer$WorkItem.run(AsyncResourceDisposer.java:254)
      	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      Without closer investigation, this sounds to me like a network glitch the plugin is not able to recover from as it is invoking a blocking operation (socketRead) without a timeout.

            pajasoft Pavel Janoušek
            olivergondza Oliver Gondža
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: