Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-66261

Kubernetes Credentials Provider plugin does not respect 'no_proxy' configurations when calling Kubernetes API

      Kubernetes Credentials Provider plugin does not respect 'no_proxy' Java property, Environment Variable or Jenkins Proxy config from 'Jenkins->Manage Jenkins->Manage Plugins->Advanced Tab'.

      We also tried with "-Dno.proxy=kubernetes.default.svc,kubernetes.default,..." parameter set with no luck.

      Note that, we're able to access Kubernetes API pod console of the Jenkins server via 'curl' with appropriate proxy environment variables set.

       

      Please find below exception message we got;

       

      Failed to initialize Kubernetes secret provider
      java.io.IOException: Failed to authenticate with proxy
      at okhttp3.internal.connection.RealConnection.createTunnel(RealConnection.java:410)
      at okhttp3.internal.connection.RealConnection.connectTunnel(RealConnection.java:220)
      at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:161)
      at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:258)
      at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
      at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
      at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:133)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at io.fabric8.kubernetes.client.utils.TokenRefreshInterceptor.intercept(TokenRefreshInterceptor.java:42)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createApplicableInterceptors$6(HttpClientUtils.java:284)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
      at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
      at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
      at okhttp3.RealCall.execute(RealCall.java:93)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:541)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:504)
      at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:487)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:163)
      Caused: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list] for kind: [Secret] with name: [null] in namespace: [jenkins] failed.
      at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
      at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:170)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:672)
      at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:86)
      at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.startWatchingForSecrets(KubernetesCredentialProvider.java:116)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
      at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
      at jenkins.model.Jenkins$5.runTask(Jenkins.java:1131)
      at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

       

          [JENKINS-66261] Kubernetes Credentials Provider plugin does not respect 'no_proxy' configurations when calling Kubernetes API

          James Nord added a comment - - edited

          just need to replicate the following code in the builder.

          https://github.com/jenkinsci/kubernetes-plugin/blob/aed016b8d357d9a90b1d84361b32e659f606fd90/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesFactoryAdapter.java#L130-L144

          I am more suprised that it even tried to use a proxy... are any system properties set to configure the proxy? (because the code currently just ignores any proxy - as why would you use a proxy to communicate with k8s   )

          James Nord added a comment - - edited just need to replicate the following code in the builder. https://github.com/jenkinsci/kubernetes-plugin/blob/aed016b8d357d9a90b1d84361b32e659f606fd90/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesFactoryAdapter.java#L130-L144 I am more suprised that it even tried to use a proxy... are any system properties set to configure the proxy? (because the code currently just ignores any proxy - as why would you use a proxy to communicate with k8s   )

          I found that adding kubernetes.default and kubernetes.default.svc to no-proxy declarations wasn't enough. I had to also had the kubernetes service's IP address. Then 502 errors stopped and secrets started being created fine.

          Brendan Holmes added a comment - I found that adding kubernetes.default and kubernetes.default.svc to no-proxy declarations wasn't enough. I had to also had the kubernetes service's IP address. Then 502 errors stopped and secrets started being created fine.

            teilo James Nord
            fatih Fatih
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: