Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50931

EC2 Instance cap is not respected

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • ec2-plugin
    • Jenkins version: 2.107.1
      ec2-plugin version: 1.39
      Java version: 8u151
      OS: Linux (4.4.0-1052-aws)
    • 1.45

      In Manage Jenkins -> Configure System -> Cloud -> Advanced, I set "Instance Cap" to 5

      I push changes to many branches (more than 5)

      In the sidebar, I see that more than 5 instances have been created.

      I expected that a maximum of 5 instances would be created.

      I am using EC2 spot instances in the ap-southeast-2 region. I have only one 'cloud' set up, and all the instances are building (different branches of) the same job.

      Please let me know if any other information would be helpful.

      (I see that this bug has been reported previously, but closed. Is this a regression? Am I doing something wrong?)

        1. 1 - Configure EC2 plugin.png
          1 - Configure EC2 plugin.png
          147 kB
        2. 2 - Multiple instances are launched.png
          2 - Multiple instances are launched.png
          234 kB
        3. 3 - Software versions.png
          3 - Software versions.png
          89 kB

          [JENKINS-50931] EC2 Instance cap is not respected

          A B added a comment -

          Further monitoring suggests that the cap seems to struggle when multiple builds enter the queue simultaneously (or close to simultaneously). It looks like perhaps the plugin doesn't keep track of instances it has requested, only instances that are in the process of starting. So, if there's a lag between request and starting, it can request too many instances.

          That's just how it appears from the outside - I haven't examined the source code. Workaround for now is to set the instance cap at about half what you actually want it to be, and hope it doesn't exceed your true desired cap by too much.

          A B added a comment - Further monitoring suggests that the cap seems to struggle when multiple builds enter the queue simultaneously (or close to simultaneously). It looks like perhaps the plugin doesn't keep track of instances it has requested, only instances that are in the process of starting. So, if there's a lag between request and starting, it can request too many instances. That's just how it appears from the outside - I haven't examined the source code. Workaround for now is to set the instance cap at about half what you actually want it to be, and hope it doesn't exceed your true desired cap by too much.

          A B added a comment -

          An update - the problem is definitely getting worse. With an instance cap of 1, I am routinely getting 10 or more slaves starting up. Only thing that has changed is AWS (I haven't changed the Jenkins configuration at all).

           

          A B added a comment - An update - the problem is definitely getting worse. With an instance cap of 1, I am routinely getting 10 or more slaves starting up. Only thing that has changed is AWS (I haven't changed the Jenkins configuration at all).  

          Ofir added a comment -

          i have the exact same issue - the instance cap is totally not respected. i set it to 1, and almost each time i run, a new instance is created.
          i use version 2.0.7 of ec2-plugin

          Ofir added a comment - i have the exact same issue - the instance cap is totally not respected. i set it to 1, and almost each time i run, a new instance is created. i use version 2.0.7 of ec2-plugin

            francisu Francis Upton
            abbm A B
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: