Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65848

Pod Concurrency Limit not always respected

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We have a handful of pod templates defined and specified "Concurrency Limits" on those pod templates. What I noticed is that sometimes this doesn't seem to get respected.

      Version of jenkins: 2.289.1 (lts)
      Version of kubernetes plugin: 1.29.6

      I did some investigation by looking at the code and using the Script Console, unfortunately this is the end of where I'm able to figure out what's going on, but it should hopefully give a good starting point to someone that has more know-how about jenkins and the kubernetes plugin.

      First I printed out the state of the KubernetesProvisioningLimits

      print(org.csanchez.jenkins.plugins.kubernetes.KubernetesProvisioningLimits.get().podTemplateCounts)
      

      this looked strange, there seemed to be a lot more values for the template id, than we have templates.

      Then I further digged into the code and looped over all nodes (of a certain template).

      for(node in Jenkins.get().getNodes()){
        if(node.name.startsWith("my-node-template")){
          print(node.name + "\n" + node.getTemplateId() + "\n")
          print(node.template.id + "\n")
          found = false
       	for(temp in node.getCloud().templates){
            if(temp.name.equals("my-node-template")){
            	print(temp.id + ":" + temp.name + "\n")
            }
            if(temp.id.equals(node.getTemplateId())){
              print(temp)
              found = true
            }
            
      	}   
          if(!found){
            print(node.template)
          }
          
      	print("\n\n")
        }
      }
      

      This shows that some nodes seem to have a different template (and templateId), but if you do node.getCloud().getTemplates() the template cannot be found.

      What could be the reason that the node have a different template than the one in the cloud linked to the same node?
      Alternatively would it make more sense to use the pod template name instead of the id for the KubernetesProvisioningLimits?

        Attachments

          Activity

          Hide
          vlatombe Vincent Latombe added a comment -

          From what you described, I am guessing your pod templates are defined using the podTemplate step in a pipeline? If so, such limits can't be applied (because these pod templates are ephemeral and unique per call to podTemplate call)

          Show
          vlatombe Vincent Latombe added a comment - From what you described, I am guessing your pod templates are defined using the podTemplate step in a pipeline? If so, such limits can't be applied (because these pod templates are ephemeral and unique per call to podTemplate call)
          Hide
          tario Patrick Ruckstuhl added a comment -

          I did a diff between the "good" pod template on a node and the pod template on a node that it didn't find in the list of podTemplates of the corresponding cloud, and the id is the only difference, everything else is identical,

          Show
          tario Patrick Ruckstuhl added a comment - I did a diff between the "good" pod template on a node and the pod template on a node that it didn't find in the list of podTemplates of the corresponding cloud, and the id is the only difference, everything else is identical,
          Hide
          tario Patrick Ruckstuhl added a comment - - edited

          No, the templates are specified in the jenkins configuration (using jcasc), they aren't dynamic from pipelines.

          Show
          tario Patrick Ruckstuhl added a comment - - edited No, the templates are specified in the jenkins configuration (using jcasc), they aren't dynamic from pipelines.
          Hide
          tario Patrick Ruckstuhl added a comment -

          The issue is related to JCASC, the id needs to be specified on JCASC as well to keep it stable, otherwise a new one is generated whenever it is loaded

          Show
          tario Patrick Ruckstuhl added a comment - The issue is related to JCASC, the id needs to be specified on JCASC as well to keep it stable, otherwise a new one is generated whenever it is loaded

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            tario Patrick Ruckstuhl
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: