Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62970

Kubernetes credentials provider should attempt reconnect on KubernetesClientException's

      If there is a network disruption that causes the k8s client to disconnect and stop listening for secrets the plugin stops updating. See the stack trace below for example.

      In the spirit of Kubernetes the plugin should keep trying to fix the problem until the issue goes away.

      Stack Trace:

      java.net.SocketException: Socket closed
      	at java.net.SocketInputStream.read(SocketInputStream.java:204)
      	at java.net.SocketInputStream.read(SocketInputStream.java:141)
      	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
      	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
      	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
      	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
      	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
      	at okio.Okio$2.read(Okio.java:140)
      	at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
      Caused: java.net.SocketTimeoutException: timeout
      	at okio.Okio$4.newTimeoutException(Okio.java:232)
      	at okio.AsyncTimeout.exit(AsyncTimeout.java:285)
      	at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)
      	at okio.RealBufferedSource.indexOf(RealBufferedSource.java:354)
      	at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:226)
      	at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:215)
      	at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189)
      	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:111)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
      	at okhttp3.RealCall.execute(RealCall.java:93)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:411)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:372)
      	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:354)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:153)
      Caused: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list]  for kind: [Secret]  with name: [null]  in namespace: [ferris-wheel]  failed.
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
      	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:157)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:620)
      	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:69)
      	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.startWatchingForSecrets(KubernetesCredentialProvider.java:108)
      	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.onClose(KubernetesCredentialProvider.java:228)
      	at io.fabric8.kubernetes.client.utils.WatcherToggle.onClose(WatcherToggle.java:56)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.closeEvent(WatchConnectionManager.java:380)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.access$700(WatchConnectionManager.java:50)
      	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onMessage(WatchConnectionManager.java:263)
      	at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:323)
      	at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219)
      	at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105)
      	at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:274)
      	at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:214)
      	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203)
      	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

          [JENKINS-62970] Kubernetes credentials provider should attempt reconnect on KubernetesClientException's

          Kyle Cronin added a comment -

          teilo wow, sorry about all the duplicates. The form said that it failed to submit because of invalid form data so I tried different variations.

          Kyle Cronin added a comment - teilo wow, sorry about all the duplicates. The form said that it failed to submit because of invalid form data so I tried different variations.

          James Nord added a comment -

          pretty sure this is now working

          James Nord added a comment - pretty sure this is now working

            kylecronin Kyle Cronin
            kylecronin Kyle Cronin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: