Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27281

Provide more control over EC2 instance spawn rate

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Minor Minor
    • ec2-plugin
    • None

      Currently the EC2 plugin attempts to create one EC2 slave per job in the queue with some small degree of ramp up (at least in so far as I can tell) - typically on the first poll one or two slaves are started, on the second as many slaves as there are builds are started.

      It would be very nice to be able to control the rate at which new slaves are created so that there is more chance to use one EC2 slave for multiple builds.

      My use case is that I'm doing Linux kernel build jobs over a range of configurations, some of which take much longer to build than others to the point where a small number of additional slaves over the quantity needed to do the long running jobs can complete the work required in the time taken for the slow configurations. With the current algorithm the plugin will generally start as many nodes as there are builds which results in greater costs. With less aggressive creation of slaves this could be mitigated for no loss in overall throughput.

      I could limit the total number of slaves but this would mean that while individual jobs would run more efficiently the EC2 plugin would not be able to add extra capacity if multiple jobs are scheduled at the same time (as is quite common).

          [JENKINS-27281] Provide more control over EC2 instance spawn rate

          Mark Brown added a comment -

          Thinking about this some more just setting a maximum number of slaves that can be created per unit time would do the trick.

          Mark Brown added a comment - Thinking about this some more just setting a maximum number of slaves that can be created per unit time would do the trick.

          Vinay Sharma added a comment -

          broonie Hi,

          I am struggling with the same issue. The EC2 instance spawn rate is very slow and I want to speed it up.Can you please help me with the same if it has been resolved for you.

          Vinay Sharma added a comment - broonie Hi, I am struggling with the same issue. The EC2 instance spawn rate is very slow and I want to speed it up.Can you please help me with the same if it has been resolved for you.

          Mark Brown added a comment -

          Sorry, I didn't - I gave up on using EC2 entirely for that application (switched to fixed nodes instead) due to this and other issues with EC2 performance in that particular application.

          Mark Brown added a comment - Sorry, I didn't - I gave up on using EC2 entirely for that application (switched to fixed nodes instead) due to this and other issues with EC2 performance in that particular application.

            francisu Francis Upton
            broonie Mark Brown
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: