• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • kubernetes-plugin
    • None
    • Jenkins 2.222.1, Kubernetes Plugin 1.25.4

      After build completes the pod in openshift is closed down, but remains in state completed and is not removed.  When downgrading to version 1.25.3 of the plugin the issue is solved, so this must be regression in the last version (maybe PR 770 ?)

          [JENKINS-62204] Build pods not removed after build finished

          Im seeing a similar issue with plain kubernetes, in the logs I see

          2020-05-17 08:35:53.960+0000 [id=43]	INFO	o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent ubuntu-z51wv
          2020-05-17 08:35:54.245+0000 [id=2762]	INFO	j.s.DefaultJnlpSlaveReceiver#channelClosed: Computer.threadPoolForRemoting [#820] for ubuntu-z51wv terminated: java.nio.channels.ClosedChannelException
          2020-05-17 08:35:54.251+0000 [id=43]	SEVERE	hudson.triggers.SafeTimerTask#run: Timer task hudson.slaves.ComputerRetentionWork@34947c3b failed
          java.util.MissingResourceException: Can't find resource for bundle org.jvnet.localizer.ResourceBundleHolder$ResourceBundleImpl, key Setting offline before shutting down
          	at java.util.ResourceBundle.getObject(ResourceBundle.java:450)
          	at java.util.ResourceBundle.getString(ResourceBundle.java:407)
          	at org.jvnet.localizer.Localizable.toString(Localizable.java:67)
          Caused: java.lang.RuntimeException: Failed to localize key=Setting offline before shutting down,args=[]
          	at org.jvnet.localizer.Localizable.toString(Localizable.java:69)
          	at org.jvnet.localizer.Localizable.toString(Localizable.java:74)
          	at hudson.slaves.OfflineCause$SimpleOfflineCause.toString(OfflineCause.java:93)
          	at org.jenkinsci.plugins.mailwatcher.WatcherComputerListener.onTemporarilyOffline(WatcherComputerListener.java:111)
          	at hudson.model.Computer.setTemporarilyOffline(Computer.java:716)
          	at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave._terminate(KubernetesSlave.java:312)
          	at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:90)
          	at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:59)
          	at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:43)
          	at hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:70)
          	at hudson.model.Queue._withLock(Queue.java:1398)
          	at hudson.model.Queue.withLock(Queue.java:1275)
          	at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:61)
          	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:91)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          

          On the kubernetes side the agent container stops but the build container remains running

          NAME             READY   STATUS    RESTARTS   AGE   IP             NODE                        NOMINATED NODE   READINESS GATES
          ubuntu-z51wv     1/2     Running   0          14m   10.83.234.63   devdocker1.daboyz.lan.com   <none>           <none>
          

          David van Laatum added a comment - Im seeing a similar issue with plain kubernetes, in the logs I see 2020-05-17 08:35:53.960+0000 [id=43] INFO o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent ubuntu-z51wv 2020-05-17 08:35:54.245+0000 [id=2762] INFO j.s.DefaultJnlpSlaveReceiver#channelClosed: Computer.threadPoolForRemoting [#820] for ubuntu-z51wv terminated: java.nio.channels.ClosedChannelException 2020-05-17 08:35:54.251+0000 [id=43] SEVERE hudson.triggers.SafeTimerTask#run: Timer task hudson.slaves.ComputerRetentionWork@34947c3b failed java.util.MissingResourceException: Can't find resource for bundle org.jvnet.localizer.ResourceBundleHolder$ResourceBundleImpl, key Setting offline before shutting down at java.util.ResourceBundle.getObject(ResourceBundle.java:450) at java.util.ResourceBundle.getString(ResourceBundle.java:407) at org.jvnet.localizer.Localizable.toString(Localizable.java:67) Caused: java.lang.RuntimeException: Failed to localize key=Setting offline before shutting down,args=[] at org.jvnet.localizer.Localizable.toString(Localizable.java:69) at org.jvnet.localizer.Localizable.toString(Localizable.java:74) at hudson.slaves.OfflineCause$SimpleOfflineCause.toString(OfflineCause.java:93) at org.jenkinsci.plugins.mailwatcher.WatcherComputerListener.onTemporarilyOffline(WatcherComputerListener.java:111) at hudson.model.Computer.setTemporarilyOffline(Computer.java:716) at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave._terminate(KubernetesSlave.java:312) at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:90) at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:59) at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:43) at hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:70) at hudson.model.Queue._withLock(Queue.java:1398) at hudson.model.Queue.withLock(Queue.java:1275) at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:61) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:91) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) On the kubernetes side the agent container stops but the build container remains running NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ubuntu-z51wv 1/2 Running 0 14m 10.83.234.63 devdocker1.daboyz.lan.com <none> <none>

          Just upgraded to 1.25.7 and it seems to have fixed it for me

          David van Laatum added a comment - Just upgraded to 1.25.7 and it seems to have fixed it for me

            Unassigned Unassigned
            bartdevriendt Bart Devriendt
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: