-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Jenkins 2.289.2
Kubernetes plugin 1.30.11
Openshift 3.11 and Openshift 4.6
Sometimes, randomly, when we try to run a job with a pod template with two containers (jnlp container and an additional one), Jenkins starts running the pipeline before the pod is ready. The result is that when the pipeline reaches the container step it fails.
18:14:20 [Pipeline] container
18:14:20 [Pipeline] {
18:14:20 [Pipeline] sh
18:14:20 java.net.ProtocolException: Expected HTTP 101 response but was '500 Internal Server Error'
18:14:20 at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229)
18:14:20 at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196)
18:14:20 at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203)
18:14:20 at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
18:14:20 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
18:14:20 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
18:14:20 at java.base/java.lang.Thread.run(Thread.java:834)
This issue happens when the second container uses a big image (around 1Gb). The jnlp container starts but the second container is not started yet, but nevertheless Jenkins starts running the pipeline.
[JENKINS-67592] Jenkins starts pipeline before the agent pod is ready
Pablo Gomez
created issue -
Pablo Gomez
made changes -
| Environment |
Original:
Jenkins 2.289.2
Kubernetes plugin 1.30.11 |
New:
Jenkins 2.289.2
Kubernetes plugin 1.30.11 Openshift 3.11 and Openshift 4.6 |
Pablo Gomez
added a comment - Yes I did, as I mentioned it is random issue. 
Florin Hillebrand
added a comment - I see, as a workaround you could let the jnlp container sleep for a few seconds. Have you tried setting readiness/liveness probes for the container in question (but I doubt this will change something)?
I would expect some logic in kubernetes-plugin or jnlp slave that would wait for the pod to be fully available. Is your jnlp agent up-to-date?
Florin Hillebrand
added a comment - I see, as a workaround you could let the jnlp container sleep for a few seconds. Have you tried setting readiness/liveness probes for the container in question (but I doubt this will change something)?
I would expect some logic in kubernetes-plugin or jnlp slave that would wait for the pod to be fully available. Is your jnlp agent up-to-date? Pablo Gomez
added a comment - We are using a custom jnlp image. But I think that logic should be in the Kubernetes plugin and not in image. The image should be Kubernetes-agnostic.
I thought about readiness probe but that won't change anything. Jenkins starts running the pipeline even if the pod is not in Running state yet.
Pablo Gomez
added a comment - We are using a custom jnlp image. But I think that logic should be in the Kubernetes plugin and not in image. The image should be Kubernetes-agnostic.
I thought about readiness probe but that won't change anything. Jenkins starts running the pipeline even if the pod is not in Running state yet. It is surprising because the plugin waits for readiness of all containers within the agent pod before letting the build use it. Have you tried with a recent version of the plugin to check whether it still happens?
Did you use `container(CONTAINER_NAME)` with the correct CONTAINER_NAME? or is a default container specified in your podTemplate?