Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64571

Fallback to on-demand instances when spot limit reached

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: ec2-plugin
    • Labels:
    • Environment:
      Linux
      Plugin version 1.56
    • Similar Issues:

      Description

      At the moment, the plugin doesn't handle the case where the spot instance limit for the AWS account is reached. In this scenario, the request to AWS just fails with the following exception:

      WARNING hudson.plugins.ec2.EC2Cloud#provision: SlaveTemplate{description='<description>', labels='<labels>'}. Exception during provisioning
      com.amazonaws.services.ec2.model.AmazonEC2Exception: Max spot instance count exceeded (Service: AmazonEC2; Status Code: 400; Error Code: MaxSpotInstanceCountExceeded; Request ID: a7b2d9e2-43fa-4954-a830-2313b7c822dc; Proxy: <proxy>)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
              at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
              at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
              at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:26040)
              at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:26007)
              at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:25996)
              at com.amazonaws.services.ec2.AmazonEC2Client.executeRequestSpotInstances(AmazonEC2Client.java:24138)
              at com.amazonaws.services.ec2.AmazonEC2Client.requestSpotInstances(AmazonEC2Client.java:24109)
              at hudson.plugins.ec2.SlaveTemplate.provisionSpot(SlaveTemplate.java:1385)
              at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:864)
              at hudson.plugins.ec2.EC2Cloud.getNewOrExistingAvailableSlave(EC2Cloud.java:693)
              at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:774)
              at hudson.plugins.ec2.util.MinimumInstanceChecker.lambda$null$11(MinimumInstanceChecker.java:114) 

       

      I think that, if the `Fallback to on-demand instances` is selected in the agent configuration, the plugin should just provision an on-demand instance instead.

        Attachments

          Activity

          Hide
          canuck1987 Tim Brown added a comment -

          > I think that, if the `Fallback to on-demand instances` is selected in the agent configuration, the plugin should just provision an on-demand instance instead.
          +1 for this. We assumed this was the behaviour and were scratching our heads why more instances weren't starting during peak times.

          Show
          canuck1987 Tim Brown added a comment - > I think that, if the `Fallback to on-demand instances` is selected in the agent configuration, the plugin should just provision an on-demand instance instead. +1 for this. We assumed this was the behaviour and were scratching our heads why more instances weren't starting during peak times.

            People

            Assignee:
            thoulen FABRIZIO MANFREDI
            Reporter:
            enricodev Enrico Pelizzon
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: