Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64571

Fallback to on-demand instances when spot limit reached

    XMLWordPrintable

Details

    • Improvement
    • Status: Fixed but Unreleased (View Workflow)
    • Major
    • Resolution: Fixed
    • ec2-plugin
    • Linux
      Plugin version 1.56

    Description

      At the moment, the plugin doesn't handle the case where the spot instance limit for the AWS account is reached. In this scenario, the request to AWS just fails with the following exception:

      WARNING hudson.plugins.ec2.EC2Cloud#provision: SlaveTemplate{description='<description>', labels='<labels>'}. Exception during provisioning
      com.amazonaws.services.ec2.model.AmazonEC2Exception: Max spot instance count exceeded (Service: AmazonEC2; Status Code: 400; Error Code: MaxSpotInstanceCountExceeded; Request ID: a7b2d9e2-43fa-4954-a830-2313b7c822dc; Proxy: <proxy>)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
              at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
              at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
              at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
              at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:26040)
              at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:26007)
              at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:25996)
              at com.amazonaws.services.ec2.AmazonEC2Client.executeRequestSpotInstances(AmazonEC2Client.java:24138)
              at com.amazonaws.services.ec2.AmazonEC2Client.requestSpotInstances(AmazonEC2Client.java:24109)
              at hudson.plugins.ec2.SlaveTemplate.provisionSpot(SlaveTemplate.java:1385)
              at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:864)
              at hudson.plugins.ec2.EC2Cloud.getNewOrExistingAvailableSlave(EC2Cloud.java:693)
              at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:774)
              at hudson.plugins.ec2.util.MinimumInstanceChecker.lambda$null$11(MinimumInstanceChecker.java:114) 

       

      I think that, if the `Fallback to on-demand instances` is selected in the agent configuration, the plugin should just provision an on-demand instance instead.

      Attachments

        Activity

          canuck1987 Tim Brown added a comment -

          > I think that, if the `Fallback to on-demand instances` is selected in the agent configuration, the plugin should just provision an on-demand instance instead.
          +1 for this. We assumed this was the behaviour and were scratching our heads why more instances weren't starting during peak times.

          canuck1987 Tim Brown added a comment - > I think that, if the `Fallback to on-demand instances` is selected in the agent configuration, the plugin should just provision an on-demand instance instead. +1 for this. We assumed this was the behaviour and were scratching our heads why more instances weren't starting during peak times.

          People

            enricodev Enrico Pelizzon
            enricodev Enrico Pelizzon
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: