Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70612

Kubernetes plugin pod templates not respecting instance cap

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • kubernetes-plugin
    • None
    • Jenkins 2.375.1
      Kubernetes plugin:3845.va_9823979a_744
      Kube version: v1.23.13

      The instanceCap functionality should be able to let us set a limit of concurrent executors of agent type (by label).


      Say you declaratively set an instanceCap: 2 for the agent label you want to spin up, e.g. test-generic-label - You would expect that a maximum of 2/2 agents of that label can be spun up and run concurrently, if any jobs ask for more agents, they will be queued and wait.

      agent { 
        kubernetes { 
          label "test-generic-label" 
          instanceCap 2              
          yamlFile "generic-pod.yaml" 
          defaultContainer "generic-image"              
        }          
      } 

      But through testing, I found that the pod template is generated based on the label and suffixed in the Kubernetes deployment style, and once that template limit is reached:

      Template for label "test-generic-label": test-generic-label-cw4nz
      test-generic-label-cw4nz template limit reached: 2/2. Cannot add  more! 

      The label is changed dynamically with another incremented suffix, then another 2 agents are scheduled:

      Template for label "test-generic-label": test-generic-label-cg3bw
      test-generic-label-cg3bw template limit: 2/2 

      So we end up with 4/4 agents in total, even though we intend a limit of 2/2 for that label

      test-generic-label-cg3bw-tmmj1      2/2     Running     0          53s
      test-generic-label-cg3bw-x67zc      2/2     Running     0          53s
      test-generic-label-cw4nz-f64nb      2/2     Running     0          78s
      test-generic-label-cw4nz-xjzlv      2/2     Running     0          78s 


      Pod templates are generated from the label, and the limit of pod templates is respected. Once that limit is reached, the plugin simply generates new templates from that label, so it doesn’t matter what limit you set.

      This looks to be bugged as it defeats the purpose of setting an instance cap on agents.

            Unassigned Unassigned
            edusan7 Edu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: