-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
Kubernetes Plugin Version:
3985.vd26d77b_2a_48a_
Expected Outcome
When draining a k8s worker node on which a `jnlp` jenkins pod is scheduled, the kubernetes plugin should be able to intercept the SIGTERM and proceed to create a replacement jnlp container.
Actual Outcome
On k8s worker node draining, the SIGTERM is received, but the plugin takes approximately 5 minutes before it attempts to launch a replacement jnlp pod.
Output
14:42:30 [Pipeline] { (SupWorld)
14:42:30 [Pipeline] sh
14:42:39 + echo I like to moveit moveit...
14:42:39 I like to moveit moveit...
14:42:40 [Pipeline] sh
14:42:40 + sleep 3600
14:43:46 jenkins-agents/test-kubernetes-plugin-pipeline-280a90f0-ac1a-j7kn3 Container jnlp was terminated (Exit Code: 143, Reason: Error)
14:43:46
14:43:46 - jnlp -- terminated (143)
14:43:46 jenkins-agents/test-kubernetes-plugin-pipeline-280a90f0-ac1a-j7kn3 Pod just failed (Reason: null, Message: null)
14:43:47 14:43:47 - jnlp -- terminated (143) 14:44:00 Cannot contact test-kubernetes-plugin-pipeline-280a90f0-ac1a-j7kn3: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@702031c3:JNLP4-connect connection from host.amazonaws.com/1.2.3.4:12345": Remote call on JNLP4-connect connection from hostname.compute-1.amazonaws.com/1.2.3.4:57571 failed. The channel is closing down or has closed down
14:48:46 Agent test-kubernetes-plugin-pipeline-280a90f0-ac1a-j7kn3 was deleted; cancelling node body
14:48:46 Could not connect to test-kubernetes-plugin-pipeline-280a90f0-ac1a-j7kn3 to send interrupt signal to process
14:48:46 [Pipeline] }
14:48:47 [Pipeline] // stage
14:48:47 [Pipeline] }
14:48:47 [Pipeline] // node
14:48:47 [Pipeline] }
14:48:47 Agent was removed
14:48:47 org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: wf23fwf-ewf23-324sdfds-2432sdfs-1234123fsdfsd
14:48:47 Retrying 14:48:47 [Pipeline] {
14:48:47 [Pipeline] node
14:48:48 Created Pod: eks-cluster-name jenkins-agents/test-kubernetes-plugin-pipeline-280a90f0-ac1a-14vtd
14:48:54 Agent test-kubernetes-plugin-pipeline-280a90f0-ac1a-14vtd is provisioned from template test-kubernetes-plugin-pipeline-280a90f0-ac1a-4d50-8b5d-eed594fba22f-4mwmn
Pipeline
def randomId=UUID.randomUUID().toString() def podLabel = "test-kubernetes-plugin-pipeline-${randomId}" node { podTemplate( cloud: 'testing', idleMinutes: 120, label: podLabel, nodeSelector: 'instance_type=jenkins', namespace: 'jenkins', yamlMergeStrategy: merge(), containers: [ containerTemplate( name: 'jnlp', image: 'jenkins/inbound-agent:3107.v665000b_51092-4-jdk17', alwaysPullImage: false, ttyEnabled: true, workingDir: '/home/jenkins/' ), ], yaml: ''' apiVersion: v1 kind: Pod spec: terminationGracePeriodSeconds: 1800 priorityClassName: jenkins-pods tolerations: - key: dedicated operator: Equal value: jenkins effect: NoSchedule ''' ){ retry(count: 2, conditions: [agent(), kubernetesAgent(), nonresumable()]) { node(podLabel){ stage('SupWorld'){ sh 'echo I like to moveit moveit...' sh 'sleep 3600' } } } }}