Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46444

Slave Node is Left in "offline" while Slave Pod is terminated

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • kubernetes-plugin
    • None

      Using Kubernetes-Plugin 0.12 to start a slave pod in Kubernetes Cluster.

      Upon job completion, slave pod receives 'Kill' signal and pod terminates.

      However, Jenkins master still shows Slave Node as active, and marks it in "offline" status.

      Expected behaviour, Jenkins master removes the node of the terminated slave pod.

      Events Received by Jenkins Slave Pod:

      2017-08-24 23:22:08 -0700 PDT   2017-08-24 23:22:08 -0700 PDT   1         us-west-2-prod-tsshv   Pod                 Normal    Scheduled   default-scheduler   Successfully assigned us-west-2-prod-tsshv to ip-10-7
2-32-128.us-west-2.compute.internal
2017-08-24 23:22:08 -0700 PDT   2017-08-24 23:22:08 -0700 PDT   1         us-west-2-prod-tsshv   Pod       spec.containers{jnlp}   Normal    Pulling   kubelet, ip-10-72-32-128.us-west-2.compute.internal   pulling
 image "jenkinsci/jnlp-slave:alpine"
2017-08-24 23:22:16 -0700 PDT   2017-08-24 23:22:16 -0700 PDT   1         us-west-2-prod-tsshv   Pod       spec.containers{jnlp}   Normal    Pulled    kubelet, ip-10-72-32-128.us-west-2.compute.internal   Success
fully pulled image "jenkinsci/jnlp-slave:alpine"
2017-08-24 23:22:16 -0700 PDT   2017-08-24 23:22:16 -0700 PDT   1         us-west-2-prod-tsshv   Pod       spec.containers{jnlp}   Normal    Created   kubelet, ip-10-72-32-128.us-west-2.compute.internal   Created
 container with docker id 5790ffa32575; Security:[seccomp=unconfined]
2017-08-24 23:22:16 -0700 PDT   2017-08-24 23:22:16 -0700 PDT   1         us-west-2-prod-tsshv   Pod       spec.containers{jnlp}   Normal    Started   kubelet, ip-10-72-32-128.us-west-2.compute.internal   Started
 container with docker id 5790ffa32575
2017-08-24 23:22:21 -0700 PDT   2017-08-24 23:22:21 -0700 PDT   1         us-west-2-prod-tsshv   Pod       spec.containers{jnlp}   Normal    Killing   kubelet, ip-10-72-32-128.us-west-2.compute.internal   Killing

       

      Jenkins Master Log

      Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Spot instances: 1
Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template: Kubernetes Pod Template
Aug 25, 2017 6:22:08 AM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Aug 25, 2017 6:22:08 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning Kubernetes Pod Template from us-west-2-prod with 1 executors. Remaining excess workload: 0
Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
INFO: Created Pod: us-west-2-prod-tsshv
Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
INFO: Waiting for Pod to be scheduled (0/100): us-west-2-prod-tsshv
Aug 25, 2017 6:22:14 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
INFO: Container is waiting us-west-2-prod-tsshv [jnlp]: ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={})
Aug 25, 2017 6:22:14 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
INFO: Waiting for Pod to be scheduled (1/100): us-west-2-prod-tsshv
Aug 25, 2017 6:22:17 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted JNLP4-connect connection #50 from /10.4.93.0:53810
Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for slave us-west-2-prod-tsshv
Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminated Kubernetes instance for slave us-west-2-prod-tsshv
Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer us-west-2-prod-tsshv
Terminated Kubernetes instance for slave us-west-2-prod-tsshv
Aug 25, 2017 6:22:21 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
WARNING: Computer.threadPoolForRemoting [#262] for us-west-2-prod-tsshv terminated
java.nio.channels.ClosedChannelException
        at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
        at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
        at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
        at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
        at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)
        at hudson.remoting.Channel.close(Channel.java:1304)
        at hudson.remoting.Channel.close(Channel.java:1272)
        at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:708)
        at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)
        at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:626)
        at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Aug 25, 2017 6:22:22 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
INFO: test #70 completed: SUCCESS
Aug 25, 2017 6:22:28 AM hudson.slaves.NodeProvisioner$2 run
INFO: Kubernetes Pod Template provisioning successfully completed. We have now 3 computer(s)

       

      After this, Jenkins nodes show the "offline" node

      I can delete the "offline" node manually via UI, upon which master log repors:

      Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for slave us-west-2-prod-tsshv
Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminated Kubernetes instance for slave us-west-2-prod-tsshv
Terminated Kubernetes instance for slave us-west-2-prod-tsshv
Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer us-west-2-prod-tsshv

        1. image-2019-03-27-16-16-22-818.png
          image-2019-03-27-16-16-22-818.png
          65 kB
        2. image-2017-08-24-23-25-48-535.png
          image-2017-08-24-23-25-48-535.png
          79 kB

          [JENKINS-46444] Slave Node is Left in "offline" while Slave Pod is terminated

          Illya Chekrygin created issue -
          Illya Chekrygin made changes -
          Description Original: Using Kubernetes-Plugin 0.12 to start a slave pod in Kubernetes Cluster.

          Upon job completion, slave pod receives 'Kill' signal and pod terminates.

          However, Jenkins master still shows Slave Node as active, and marks it in "offline" status.

          Expected behaviour, Jenkins master removes the node of the terminated slave pod.
          ----
          h2. Events Received by Jenkins Slave Pod:

          {{2017-08-24 23:22:08 -0700 PDT 2017-08-24 23:22:08 -0700 PDT 1 us-west-2-prod-tsshv Pod Normal Scheduled default-scheduler Successfully assigned us-west-2-prod-tsshv to ip-10-72-32-128.us-west-2.compute.internal}}
           {{2017-08-24 23:22:08 -0700 PDT 2017-08-24 23:22:08 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers\{jnlp} Normal Pulling kubelet, ip-10-72-32-128.us-west-2.compute.internal pulling image "jenkinsci/jnlp-slave:alpine"}}
           {{2017-08-24 23:22:16 -0700 PDT 2017-08-24 23:22:16 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers\{jnlp} Normal Pulled kubelet, ip-10-72-32-128.us-west-2.compute.internal Successfully pulled image "jenkinsci/jnlp-slave:alpine"}}
           {{2017-08-24 23:22:16 -0700 PDT 2017-08-24 23:22:16 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers\{jnlp} Normal Created kubelet, ip-10-72-32-128.us-west-2.compute.internal Created container with docker id 5790ffa32575; Security:[seccomp=unconfined]}}
           {{2017-08-24 23:22:16 -0700 PDT 2017-08-24 23:22:16 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers\{jnlp} Normal Started kubelet, ip-10-72-32-128.us-west-2.compute.internal Started container with docker id 5790ffa32575}}
           {{2017-08-24 23:22:21 -0700 PDT 2017-08-24 23:22:21 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers\{jnlp} Normal Killing kubelet, ip-10-72-32-128.us-west-2.compute.internal Killing container with docker id 5790ffa32575: Need to kill pod.}}

           
          h2. Jenkins Master Log

          {{ug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision}}
           {{INFO: Excess workload after pending Spot instances: 1}}
           {{Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision}}
           {{INFO: Template: Kubernetes Pod Template}}
           {{Aug 25, 2017 6:22:08 AM okhttp3.internal.platform.Platform log}}
           {{INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?}}
           {{Aug 25, 2017 6:22:08 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply}}
           {{INFO: Started provisioning Kubernetes Pod Template from us-west-2-prod with 1 executors. Remaining excess workload: 0}}
           {{Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call}}
           {{INFO: Created Pod: us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call}}
           {{INFO: Waiting for Pod to be scheduled (0/100): us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:22:14 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call}}
           {{INFO: Container is waiting us-west-2-prod-tsshv [jnlp]: ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties=\{})}}
           {{Aug 25, 2017 6:22:14 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call}}
           {{INFO: Waiting for Pod to be scheduled (1/100): us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:22:17 AM hudson.TcpSlaveAgentListener$ConnectionHandler run}}
           {{INFO: Accepted JNLP4-connect connection #50 from /10.4.93.0:53810}}
           {{Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate}}
           {{INFO: Terminating Kubernetes instance for slave us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate}}
           {{INFO: Terminated Kubernetes instance for slave us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate}}
           {{INFO: Disconnected computer us-west-2-prod-tsshv}}
           {{Terminated Kubernetes instance for slave us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:22:21 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed}}
           {{WARNING: Computer.threadPoolForRemoting [#262] for us-west-2-prod-tsshv terminated}}
           {{java.nio.channels.ClosedChannelException}}
           {{at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)}}
           {{at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)}}
           {{at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)}}
           {{at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)}}
           {{at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)}}
           {{at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)}}
           {{at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)}}
           {{at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)}}
           {{at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)}}
           {{at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)}}
           {{at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)}}
           {{at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)}}
           {{at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)}}
           {{at hudson.remoting.Channel.close(Channel.java:1304)}}
           {{at hudson.remoting.Channel.close(Channel.java:1272)}}
           {{at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:708)}}
           {{at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)}}
           {{at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:626)}}
           {{at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)}}
           {{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
           {{at java.util.concurrent.FutureTask.run(FutureTask.java:266)}}
           {{at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
           {{at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}{{at java.lang.Thread.run(Thread.java:748)}}{{Aug 25, 2017 6:22:22 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finish}}
           {{INFO: test #70 completed: SUCCESS}}
           {{Aug 25, 2017 6:22:28 AM hudson.slaves.NodeProvisioner$2 run}}
           {{INFO: Kubernetes Pod Template provisioning successfully completed. We have now 3 computer(s)}}

           

          After this, Jenkins nodes show the "offline" node

          !image-2017-08-24-23-25-48-535.png!

          I can delete the "offline" node manually via UI, upon which master log repors:

          {{Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate}}
           {{INFO: Terminating Kubernetes instance for slave us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate}}
           {{INFO: Terminated Kubernetes instance for slave us-west-2-prod-tsshv}}
           {{Terminated Kubernetes instance for slave us-west-2-prod-tsshv}}
           {{Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate}}
           {{INFO: Disconnected computer us-west-2-prod-tsshv}}

                      		New: Using Kubernetes-Plugin 0.12 to start a slave pod in Kubernetes Cluster.

          Upon job completion, slave pod receives 'Kill' signal and pod terminates.

          However, Jenkins master still shows Slave Node as active, and marks it in "offline" status.

          Expected behaviour, Jenkins master removes the node of the terminated slave pod.
          ----
          h2. Events Received by Jenkins Slave Pod:

          {code:java}
          2017-08-24 23:22:08 -0700 PDT 2017-08-24 23:22:08 -0700 PDT 1 us-west-2-prod-tsshv Pod Normal Scheduled default-scheduler Successfully assigned us-west-2-prod-tsshv to ip-10-7
          2-32-128.us-west-2.compute.internal
          2017-08-24 23:22:08 -0700 PDT 2017-08-24 23:22:08 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers{jnlp} Normal Pulling kubelet, ip-10-72-32-128.us-west-2.compute.internal pulling
           image "jenkinsci/jnlp-slave:alpine"
          2017-08-24 23:22:16 -0700 PDT 2017-08-24 23:22:16 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers{jnlp} Normal Pulled kubelet, ip-10-72-32-128.us-west-2.compute.internal Success
          fully pulled image "jenkinsci/jnlp-slave:alpine"
          2017-08-24 23:22:16 -0700 PDT 2017-08-24 23:22:16 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers{jnlp} Normal Created kubelet, ip-10-72-32-128.us-west-2.compute.internal Created
           container with docker id 5790ffa32575; Security:[seccomp=unconfined]
          2017-08-24 23:22:16 -0700 PDT 2017-08-24 23:22:16 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers{jnlp} Normal Started kubelet, ip-10-72-32-128.us-west-2.compute.internal Started
           container with docker id 5790ffa32575
          2017-08-24 23:22:21 -0700 PDT 2017-08-24 23:22:21 -0700 PDT 1 us-west-2-prod-tsshv Pod spec.containers{jnlp} Normal Killing kubelet, ip-10-72-32-128.us-west-2.compute.internal Killing
          {code}
           
          h2. Jenkins Master Log

          {code:java}
          Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
          INFO: Excess workload after pending Spot instances: 1
          Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
          INFO: Template: Kubernetes Pod Template
          Aug 25, 2017 6:22:08 AM okhttp3.internal.platform.Platform log
          INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
          Aug 25, 2017 6:22:08 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
          INFO: Started provisioning Kubernetes Pod Template from us-west-2-prod with 1 executors. Remaining excess workload: 0
          Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
          INFO: Created Pod: us-west-2-prod-tsshv
          Aug 25, 2017 6:22:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
          INFO: Waiting for Pod to be scheduled (0/100): us-west-2-prod-tsshv
          Aug 25, 2017 6:22:14 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
          INFO: Container is waiting us-west-2-prod-tsshv [jnlp]: ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={})
          Aug 25, 2017 6:22:14 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call
          INFO: Waiting for Pod to be scheduled (1/100): us-west-2-prod-tsshv
          Aug 25, 2017 6:22:17 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
          INFO: Accepted JNLP4-connect connection #50 from /10.4.93.0:53810
          Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
          INFO: Terminating Kubernetes instance for slave us-west-2-prod-tsshv
          Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
          INFO: Terminated Kubernetes instance for slave us-west-2-prod-tsshv
          Aug 25, 2017 6:22:21 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
          INFO: Disconnected computer us-west-2-prod-tsshv
          Terminated Kubernetes instance for slave us-west-2-prod-tsshv
          Aug 25, 2017 6:22:21 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
          WARNING: Computer.threadPoolForRemoting [#262] for us-west-2-prod-tsshv terminated
          java.nio.channels.ClosedChannelException
                  at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
                  at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
                  at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
                  at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
                  at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
                  at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
                  at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
                  at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
                  at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
                  at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
                  at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
                  at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
                  at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)
                  at hudson.remoting.Channel.close(Channel.java:1304)
                  at hudson.remoting.Channel.close(Channel.java:1272)
                  at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:708)
                  at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)
                  at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:626)
                  at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                  at java.lang.Thread.run(Thread.java:748)

          Aug 25, 2017 6:22:22 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
          INFO: test #70 completed: SUCCESS
          Aug 25, 2017 6:22:28 AM hudson.slaves.NodeProvisioner$2 run
          INFO: Kubernetes Pod Template provisioning successfully completed. We have now 3 computer(s)
          {code}
           

          After this, Jenkins nodes show the "offline" node

          !image-2017-08-24-23-25-48-535.png!

          I can delete the "offline" node manually via UI, upon which master log repors:

          {code:java}
          Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
          INFO: Terminating Kubernetes instance for slave us-west-2-prod-tsshv
          Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
          INFO: Terminated Kubernetes instance for slave us-west-2-prod-tsshv
          Terminated Kubernetes instance for slave us-west-2-prod-tsshv
          Aug 25, 2017 6:26:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
          INFO: Disconnected computer us-west-2-prod-tsshv
          {code}
          Carlos Sanchez made changes -
          Link New: This issue duplicates JENKINS-35246 [ JENKINS-35246 ]
          Carlos Sanchez made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Closed [ 6 ]
          kai kai made changes -
          Attachment New: image-2019-03-27-16-16-22-818.png [ 46568 ]

            csanchez Carlos Sanchez
            ichekrygin Illya Chekrygin
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: