-
Bug
-
Resolution: Unresolved
-
Major
-
None
Kubernetes Environment - 1.14.7
Jenkins Kubernetes Plugin - 1.31.3
We have noticed after the update of the plugin from 1.31.2 to 1.31.3, the pods that get created are twice or more as a part of the plugin interaction with Kubernetes.
This happens often but sometimes it just creates a single pod as requested and moves on. In the below example screenshot, you can see similar pods getting created in a span of 30 or more seconds.(see the time) What could be causing this?
We use both pod templates that is populated in the configure cloud section and also we generate the pod manifest yaml dynamically based on labels. This occurs in both the cases.
We also noticed that pod is being retained for 5 minutes or so before terminating. We double checked the pod retention policy is set as 'never' at global level.
Thanks for your time.
We have a similar problem with Kubernetes Environment - 1.12.10 + Jenkins Kubernetes Plugin - 1.31.3. Normally we get one pod per job, but it has happened twice to us when each job would create more than 100 pods, and the pods are spaced 30s minimum and upto 5 minutes apart. Once getting this mode, after a few hours the kubernetes cluster would be hosed with thousands of exited containers per worker node. We would have to restore the kubenetes worker nodes from backups to get rid of the exited containers quickly while also clear the job queues in jenkins.
It would be desirable to configure the number of pods each job can create and let the job to fail once the limit is reached. It would also be desirable to configure more time between pods.