• Icon: New Feature New Feature
    • Resolution: Fixed
    • Icon: Major Major
    • ec2-plugin
    • None

      The current limits in the EC2 plugin bases the limits upon overal EC2 instances. In environments where EC2 is used for things other than Jenkins build slaves, this kind of limit is not useful. There may be any number of instances running in an EC2 account that are not related to performing builds and should not be counted towards this limit.

      The EC2 plugin should only count the instances that it creates and destroys, likely by tracking instance IDs.

      Ideally there would be an overall global limit set, but the plugin doesn't currently have a global config section, so that may be prohibitively complicated to implement to support this one feature. With a global limit as well as per-cloud limits, one could set a global limit of say 15, and then have two clouds with a limit of 10, so each one could spin up as many as 10 boxes. If one already has 10 going though the other gets stopped at 5.

      Looking at the existing structure, it may be better to improve the Cloud-level limit so that it (optionally with a check box?) tracks only build hosts, and then add the per-AMI limits as suggested in JENKINS-6676, with the global limit being added at a later date as more global config options are needed.

          [JENKINS-14965] Instance limits improvement

          max allan added a comment -

          I like the idea of the node cap applying only to instances that Jenkins has started itself. Simply ignore any other instances.

          max allan added a comment - I like the idea of the node cap applying only to instances that Jenkins has started itself. Simply ignore any other instances.

          Jose Sa added a comment -

          It would make some sense to make the plugin check for maximum limits on different levels:

          • Globally (current behavior): check for limit of instances deployed for the same user in that cloud
          • Locally: check for limit of instances deployed by current Jenkins master
          • Instance: check for limit of instances deployed for a particular AMI

          Minimum limits could also be defined on Instance level if needed to have a small starting set of instances and expand to more on load demand.

          Jose Sa added a comment - It would make some sense to make the plugin check for maximum limits on different levels: Globally (current behavior): check for limit of instances deployed for the same user in that cloud Locally: check for limit of instances deployed by current Jenkins master Instance: check for limit of instances deployed for a particular AMI Minimum limits could also be defined on Instance level if needed to have a small starting set of instances and expand to more on load demand.

          Daniel Aquino added a comment -

          Instances can be tagged so it would be nice if we could set the instance cap based on the tag we give it..

          Daniel Aquino added a comment - Instances can be tagged so it would be nice if we could set the instance cap based on the tag we give it..

          Daniel Aquino added a comment - - edited

          Just noticed that progress has started on this and I'd like to reiterate my point that I believe limiting based on instance tags sounds like the logical way to control this since someone would want to individually control each host configuration that's created.. For I may want a generic slave host profile with max 30 instances while I have another unit testing profile that I only want to spin up a max of 5.

          Daniel Aquino added a comment - - edited Just noticed that progress has started on this and I'd like to reiterate my point that I believe limiting based on instance tags sounds like the logical way to control this since someone would want to individually control each host configuration that's created.. For I may want a generic slave host profile with max 30 instances while I have another unit testing profile that I only want to spin up a max of 5.

          Jan Van Besien added a comment - - edited

          I added a very minimal patch which simply tags the jenkins slaves and only looks at tagged ec2 instances when counting the number of already running instances.

          This is still not perfect (e.g. what if you have multiple jenkins instances) but IMHO definitely already more useful than the current standard behavior.

          Jan Van Besien added a comment - - edited I added a very minimal patch which simply tags the jenkins slaves and only looks at tagged ec2 instances when counting the number of already running instances. This is still not perfect (e.g. what if you have multiple jenkins instances) but IMHO definitely already more useful than the current standard behavior.

          Daniel Aquino added a comment -

          Jan, that's great and I definitely agree that it's better than where we are right now.

          I think it would be better though if we could take advantage of the fact that each AMI has it's own set of Tags and Instance cap! (screenshot here: http://screencast.com/t/zYMzFGti)

          I really picture desiring this per AMI config because I may have AMIs that are generic for any job that could spin up dozens of slaves while others that should really only spin up one or two..

          What do you think ?

          Daniel Aquino added a comment - Jan, that's great and I definitely agree that it's better than where we are right now. I think it would be better though if we could take advantage of the fact that each AMI has it's own set of Tags and Instance cap! (screenshot here: http://screencast.com/t/zYMzFGti ) I really picture desiring this per AMI config because I may have AMIs that are generic for any job that could spin up dozens of slaves while others that should really only spin up one or two.. What do you think ?

          Francis Upton added a comment -

          The instances are now tagged, so this should resolve this.

          Francis Upton added a comment - The instances are now tagged, so this should resolve this.

            francisu Francis Upton
            qhartman Quentin Hartman
            Votes:
            3 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: