Hi,

After upgrade of Jenkins LTS 2.426.3 with helm I could observe that there's a timeout waiting for a Kubernetes Pod with the name 'jenkins-slave-j82rz' to be created in the 'jenkins' namespace. This Pod is part of a Jenkins agent launched using the Kubernetes plugin.

Kubernetes plugin I am using: 4186.v1d804571d5d4
EKS Cluster's K8s version: 1.21

Please help me to fix this connectivity issue with k8s pods. Thanks.

System log:
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePod
Terminated Kubernetes instance for agent jenkins/jenkins-slave-lt9lz
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Disconnected computer jenkins-slave-lt9lz
Feb 05, 2024 6:46:01 AM INFO org.jenkinsci.plugins.cloudstats.CloudStatistics getIdFor
No support for cloud-stats-plugin by class org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnPodFailed onEvent
jenkins/jenkins-slave-lt9lz Pod just failed. Removing the corresponding Jenkins agent. Reason: null, Message: null
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-lt9lz
Feb 05, 2024 6:46:01 AM SEVERE org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Computer for agent is null: jenkins-slave-lt9lz
Feb 05, 2024 6:46:01 AM INFO hudson.slaves.AbstractCloudSlave terminate
FATAL: Computer for agent is null: jenkins-slave-lt9lz
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnContainerTerminated lambda$onEvent$1
jenkins/jenkins-slave-fvzrd Container jnlp was just terminated, so removing the corresponding Jenkins agent
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-fvzrd
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePod
Terminated Kubernetes instance for agent jenkins/jenkins-slave-fvzrd
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Disconnected computer jenkins-slave-fvzrd
Feb 05, 2024 6:46:01 AM INFO org.jenkinsci.plugins.cloudstats.CloudStatistics getIdFor
No support for cloud-stats-plugin by class org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnPodFailed onEvent
jenkins/jenkins-slave-fvzrd Pod just failed. Removing the corresponding Jenkins agent. Reason: null, Message: null
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-fvzrd
Feb 05, 2024 6:46:01 AM SEVERE org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Computer for agent is null: jenkins-slave-fvzrd
Feb 05, 2024 6:46:01 AM INFO hudson.slaves.AbstractCloudSlave terminate
FATAL: Computer for agent is null: jenkins-slave-fvzrd
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnContainerTerminated lambda$onEvent$1
jenkins/jenkins-slave-b2zf3 Container jnlp was just terminated, so removing the corresponding Jenkins agent
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-b2zf3
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePod
Terminated Kubernetes instance for agent jenkins/jenkins-slave-b2zf3
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Disconnected computer jenkins-slave-b2zf3
Feb 05, 2024 6:46:01 AM INFO org.jenkinsci.plugins.cloudstats.CloudStatistics getIdFor
No support for cloud-stats-plugin by class org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnPodFailed onEvent
jenkins/jenkins-slave-b2zf3 Pod just failed. Removing the corresponding Jenkins agent. Reason: null, Message: null
Feb 05, 2024 6:46:01 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-b2zf3
Feb 05, 2024 6:46:01 AM SEVERE org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Computer for agent is null: jenkins-slave-b2zf3
Feb 05, 2024 6:46:01 AM INFO hudson.slaves.AbstractCloudSlave terminate
FATAL: Computer for agent is null: jenkins-slave-b2zf3
Feb 05, 2024 6:46:02 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnContainerTerminated lambda$onEvent$1
jenkins/jenkins-slave-8dlxt Container jnlp was just terminated, so removing the corresponding Jenkins agent
Feb 05, 2024 6:46:02 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-8dlxt
Feb 05, 2024 6:46:02 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePod
Terminated Kubernetes instance for agent jenkins/jenkins-slave-8dlxt
Feb 05, 2024 6:46:02 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Disconnected computer jenkins-slave-8dlxt
Feb 05, 2024 6:46:02 AM INFO org.jenkinsci.plugins.cloudstats.CloudStatistics getIdFor
No support for cloud-stats-plugin by class org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave
Feb 05, 2024 6:46:02 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnPodFailed onEvent
jenkins/jenkins-slave-8dlxt Pod just failed. Removing the corresponding Jenkins agent. Reason: null, Message: null
Feb 05, 2024 6:46:02 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-8dlxt
Feb 05, 2024 6:46:02 AM SEVERE org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Computer for agent is null: jenkins-slave-8dlxt
Feb 05, 2024 6:46:02 AM INFO hudson.slaves.AbstractCloudSlave terminate
FATAL: Computer for agent is null: jenkins-slave-8dlxt
Feb 05, 2024 6:46:03 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnContainerTerminated lambda$onEvent$1
jenkins/jenkins-slave-w6ch8 Container jnlp was just terminated, so removing the corresponding Jenkins agent
Feb 05, 2024 6:46:03 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-w6ch8
Feb 05, 2024 6:46:03 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePod
Terminated Kubernetes instance for agent jenkins/jenkins-slave-w6ch8
Feb 05, 2024 6:46:03 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Disconnected computer jenkins-slave-w6ch8
Feb 05, 2024 6:46:03 AM INFO org.jenkinsci.plugins.cloudstats.CloudStatistics getIdFor
No support for cloud-stats-plugin by class org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave
Feb 05, 2024 6:46:03 AM INFO org.csanchez.jenkins.plugins.kubernetes.pod.retention.Reaper$TerminateAgentOnPodFailed onEvent
jenkins/jenkins-slave-w6ch8 Pod just failed. Removing the corresponding Jenkins agent. Reason: null, Message: null
Feb 05, 2024 6:46:03 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Terminating Kubernetes instance for agent jenkins-slave-w6ch8
Feb 05, 2024 6:46:03 AM SEVERE org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
Computer for agent is null: jenkins-slave-w6ch8
Feb 05, 2024 6:46:03 AM INFO hudson.slaves.AbstractCloudSlave terminate
FATAL: Computer for agent is null: jenkins-slave-w6ch8
Feb 05, 2024 6:46:09 AM INFO hudson.slaves.NodeProvisioner update
jenkins-slave-mrl3x provisioning successfully completed. We have now 3 computer(s)
Feb 05, 2024 6:46:09 AM WARNING org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
Error in provisioning; agent=KubernetesSlave name: jenkins-slave-hxkkn, template=PodTemplate{id='520dc4e6-e0df-45fb-b462-589002329332', name='jenkins-slave', slaveConnectTimeout=100, label='jenkins-agent', serviceAccount='default', nodeUsageMode=NORMAL, podRetention='Never', volumes=[HostPathVolume [mountPath=/var/run/docker.sock, hostPath=/var/run/docker.sock]], containers=[ContainerTemplate{name='jnlp', image='gcr.io/blackjack-209019/pipeline/dev-build/build-engine-autoscaling/sofy-jenkins-agent:latest', workingDir='/home/jenkins', command='', args='', resourceRequestCpu='', resourceRequestMemory='', resourceRequestEphemeralStorage='', resourceLimitCpu='', resourceLimitMemory='', resourceLimitEphemeralStorage='', envVars=[KeyValueEnvVar [getValue()=https://jenkins.hclpnpsofy.com/, getKey()=JENKINS_URL]], livenessProbe=ContainerLivenessProbe{execArgs='', timeoutSeconds=0, initialDelaySeconds=0, failureThreshold=0, periodSeconds=0, successThreshold=0}}], imagePullSecrets=[PodImagePullSecret\{name='gcr-json-key'}]} io.fabric8.kubernetes.client.KubernetesClientTimeoutException: Timed out waiting for [100000] milliseconds for [Pod] with name:[jenkins-slave-hxkkn] in namespace [jenkins]. at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.waitUntilCondition(BaseOperation.java:939) at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.waitUntilReady(BaseOperation.java:921) at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.waitUntilReady(BaseOperation.java:97) at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:185) at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:297) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)