Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62970

Kubernetes credentials provider should attempt reconnect on KubernetesClientException's

XMLWordPrintable

      If there is a network disruption that causes the k8s client to disconnect and stop listening for secrets the plugin stops updating. See the stack trace below for example.

      In the spirit of Kubernetes the plugin should keep trying to fix the problem until the issue goes away.

      Stack Trace:

      java.net.SocketException: Socket closed
      	at java.net.SocketInputStream.read(SocketInputStream.java:204)
      	at java.net.SocketInputStream.read(SocketInputStream.java:141)
      	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
      	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
      	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
      	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
      	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
      	at okio.Okio$2.read(Okio.java:140)
      	at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
      Caused: java.net.SocketTimeoutException: timeout
      	at okio.Okio$4.newTimeoutException(Okio.java:232)
      	at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
      	at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
      	at okio.RealBufferedSource.indexOf(RealBufferedSource.java:354)
      	at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:226)
      	at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:215)
      	at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
      	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:111)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
      	at okhttp3.RealCall.execute(RealCall.java:93)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:411)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:372)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:354)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:153)
      Caused: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list]  for kind: [Secret]  with name: [null]  in namespace: [ferris-wheel]  failed.
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:157)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:620)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:69)
      	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.startWatchingForSecrets(KubernetesCredentialProvider.java:108)
      	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.onClose(KubernetesCredentialProvider.java:228)
      	at io.fabric8.kubernetes.client.utils.WatcherToggle.onClose(WatcherToggle.java:56)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.closeEvent(WatchConnectionManager.java:380)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.access$700(WatchConnectionManager.java:50)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onMessage(WatchConnectionManager.java:263)
      	at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:323)
      	at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219)
      	at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105)
      	at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:274)
      	at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:214)
      	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203)
      	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

            kylecronin Kyle Cronin
            kylecronin Kyle Cronin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: