Details
-
Type:
Bug
-
Status: Closed (View Workflow)
-
Priority:
Critical
-
Resolution: Fixed
-
Component/s: kubernetes-plugin
-
Labels:None
-
Environment:Kubernetes v1.15.3
-
Similar Issues:
-
Released As:1.18.2
Description
I've upgraded a Kubernetes cluster from v1.15.2 to v1.15.3 and started noticing kubernetes-plugin cannot provision any pods. I was able to rollback to fix the issue, roll forward again to cause it, and generally isolate it to the specific cluster version bump. I can change flip just the apiserver version back and forth to cause and fix the issue. I've tried both kubernetes-plugin v1.16.4 and v1.18.1 (both affected).
What may be relevant is that Kubernetes v1.13.10, v1.14.6, and v1.15.3 was published today with CVE fixes for Go net/http vulnerabilities. Maybe one of those fixes interferes with kubernetes-plugin operation (that plugin code seems to concern the websocket and HTTP watch). I haven't confirmed the issue is present on Kubernetes v1.13.10 or v1.14.6, but suspect its on all the patched releases.
Error in provisioning; agent=KubernetesSlave name: python-pod-0ppnb-qnvc7, template=PodTemplate{, name='python-pod-0ppnb', namespace='jenkins', label='python-pod', nodeUsageMode=EXCLUSIVE, containers=[ContainerTemplate{name='python-agent', image='quay.io/dghubble/python-agent:4462978780179489a5608c480d9c64f8cb61bc7f', workingDir='/home/jenkins/agent', command='cat', ttyEnabled=true}], annotations=[org.csanchez.jenkins.plugins.kubernetes.PodAnnotation@aab9c821]}
io.fabric8.kubernetes.client.KubernetesClientException:
at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onFailure(WatchConnectionManager.java:198)
at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:571)
at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:198)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#changelog-since-v1152
Attachments
Issue Links
- is duplicated by
-
-
- Closed
-
Activity
Hi Carlos, what is this pull request going to do for us? The work around mentioned in the thread currently works for us which is https://kubernetes.default:443 in the plugin.
Ahuhu uhuhuh
added a comment - Hi Carlos, what is this pull request going to do for us? The work around mentioned in the thread currently works for us which is https://kubernetes.default:443 in the plugin. The pr avoids the need for the workaround
carlos is the problem fixed? can we download the latest plugin that has the problems fixed?
Ahuhu uhuhuh
added a comment - carlos is the problem fixed? can we download the latest plugin that has the problems fixed? Was having the same issue this morning, upgrading the plug-in to the most recent version (1.18.3) fixed the issue and doesn't require the port workaround!
Kote Mushegiani
added a comment - Was having the same issue this morning, upgrading the plug-in to the most recent version (1.18.3) fixed the issue and doesn't require the port workaround! 
PR at https://github.com/jenkinsci/kubernetes-plugin/pull/582