Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62968

Kubernetes credentials provider should attempt reconnect on KubernetesClientException's

      If there is a network disruption that causes the k8s client to disconnect and stop listening for secrets the plugin stops updating. See the stack trace below for example.

      In the spirit of Kubernetes the plugin should keep trying to fix the problem until the issue goes away.

      Stack Trace:

      java.net.SocketException: Socket closed
      	at java.net.SocketInputStream.read(SocketInputStream.java:204)
      	at java.net.SocketInputStream.read(SocketInputStream.java:141)
      	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
      	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
      	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
      	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
      	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
      	at okio.Okio$2.read(Okio.java:140)
      	at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
      Caused: java.net.SocketTimeoutException: timeout
      	at okio.Okio$4.newTimeoutException(Okio.java:232)
      	at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
      	at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
      	at okio.RealBufferedSource.indexOf(RealBufferedSource.java:354)
      	at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:226)
      	at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:215)
      	at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
      	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:111)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
      	at okhttp3.RealCall.execute(RealCall.java:93)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:411)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:372)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:354)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:153)
      Caused: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list]  for kind: [Secret]  with name: [null]  in namespace: [ferris-wheel]  failed.
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:157)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:620)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:69)
      	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.startWatchingForSecrets(KubernetesCredentialProvider.java:108)
      	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.onClose(KubernetesCredentialProvider.java:228)
      	at io.fabric8.kubernetes.client.utils.WatcherToggle.onClose(WatcherToggle.java:56)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.closeEvent(WatchConnectionManager.java:380)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.access$700(WatchConnectionManager.java:50)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onMessage(WatchConnectionManager.java:263)
      	at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:323)
      	at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219)
      	at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105)
      	at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:274)
      	at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:214)
      	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203)
      	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

          [JENKINS-62968] Kubernetes credentials provider should attempt reconnect on KubernetesClientException's

          James Nord added a comment -

          pretty sure this is fixed now

          James Nord added a comment - pretty sure this is fixed now

          James Nord added a comment -

          pretty sure this is now working

          James Nord added a comment - pretty sure this is now working

            Unassigned Unassigned
            kylecronin Kyle Cronin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: