Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60249

Pod retention policy onFailure is not applied

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • kubernetes-plugin
    • None
    • Kubernetes v1.14
      Helm 2.15.1
      Kubernetes plugin v1.21.3
      Jenkins v2.190.3

      When the Pod retention policy is set on "onFailure" and the build fails, the slaves should be kept for debug and analysis. However, they are terminated immediately as if the build was succesful.

       

      For instance, we can use this simple Pod template.

      podTemplate(podRetention: onFailure(),
      activeDeadlineSeconds: 600,
      yaml: """
      apiVersion: v1
      kind: Pod
      spec:
       containers:
       - name: ansible
       image: (private repo)/ansible:0.2
       imagePullPolicy: IfNotPresent
       command: ['cat']
       tty: true
      """
      )

       

      I have also used a simple pipeline

      {
          node(POD_LABEL) {
            container('ansible') {
              sh "sleep 10"
              sh "hola"
            }
          }
      }
      

      As a result the pod will fail, since the if not such command as "hola".

      Due to the Pod retention policy, I would expect that the pod is still alive for the timeout configured, but Jenkins terminates it.

      To reproduce this issue is as simple as deploy the stable/jenkins Helm chart and create a pipeline with the details given above.

          [JENKINS-60249] Pod retention policy onFailure is not applied

          Can I provide more information to clarify?

           

          Ignacio Pascual added a comment - Can I provide more information to clarify?  

          elhay efrat added a comment -

          ijpascual 

          according to the code

          public boolean shouldDeletePod(KubernetesCloud cloud, Pod pod) {
          if (pod == null || pod.getStatus() == null)

          { return false; }

          boolean hasErrors = pod.getStatus().getPhase().toLowerCase().matches("(failed|unknown)");
          return !hasErrors;
          }

           

           

          it will keep the pods only if the pods are in failure or unknow state 

          not the job  

          elhay efrat added a comment - ijpascual   according to the code public boolean shouldDeletePod(KubernetesCloud cloud, Pod pod) { if (pod == null || pod.getStatus() == null) { return false; } boolean hasErrors = pod.getStatus().getPhase().toLowerCase().matches("(failed|unknown)"); return !hasErrors; }     it will keep the pods only if the pods are in failure or unknow state  not the job  

          Thanks for your response.

          I see. In that case, Jenkins does not keep the pods when there are pipeline execution failures, but when there is some failure with the pod itself.

          According to the documentation, it should take the status of the build. Right?

          https://jenkins.io/doc/pipeline/steps/kubernetes/

           

          Ignacio Pascual added a comment - Thanks for your response. I see. In that case, Jenkins does not keep the pods when there are pipeline execution failures, but when there is some failure with the pod itself. According to the documentation, it should take the status of the build. Right? https://jenkins.io/doc/pipeline/steps/kubernetes/  

            Unassigned Unassigned
            ijpascual Ignacio Pascual
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: