Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35708

EC2 Plugin: Ability to round robin EC2 availability zones

    XMLWordPrintable

Details

    • Improvement
    • Status: Fixed but Unreleased (View Workflow)
    • Minor
    • Resolution: Fixed
    • ec2-plugin
    • None
    • 1.41

    Description

      Is it possible to have instances to spin up across several availability zones (a, b, c, d) to help maximize the availability of creating spot instance in zones that may have greater availability. We have tried to setup different clouds and different instances with different availability zones, but with the same label only to find it takes the first occurrence in the configuration.

      Attachments

        Activity

          nforbes Nicola Forbes added a comment - - edited

          Hi,

          Is this feature now live?  Are there any instructions on how to use it?  I've tried providing a comma-separated list of subnets in different AZs but receive error
          "com.amazonaws.AmazonClientException: Security groups must all be VPC security groups to work in a VPC context". I can only launch slaves if I specify one subnet and the related AZ.

          nforbes Nicola Forbes added a comment - - edited Hi, Is this feature now live?  Are there any instructions on how to use it?  I've tried providing a comma-separated list of subnets in different AZs but receive error "com.amazonaws.AmazonClientException: Security groups must all be VPC security groups to work in a VPC context". I can only launch slaves if I specify one subnet and the related AZ.
          davidfluck David Fluck added a comment -

          It looks like this change has indeed been released. I've also encountered that error before. How are you listing the subnets? Are you using subnet IDs? Are all of the subnets in the same VPC?

          This is the relevant code:

          private List<String> getEc2SecurityGroups(AmazonEC2 ec2) throws AmazonClientException {
              List<String> groupIds = new ArrayList<String>();
          
              DescribeSecurityGroupsResult groupResult = getSecurityGroupsBy("group-name", securityGroupSet, ec2);
              if (groupResult.getSecurityGroups().size() == 0) {
                  groupResult = getSecurityGroupsBy("group-id", securityGroupSet, ec2);
              }
          
              for (SecurityGroup group : groupResult.getSecurityGroups()) {
                  if (group.getVpcId() != null && !group.getVpcId().isEmpty()) {
                      List<Filter> filters = new ArrayList<Filter>();
                      filters.add(new Filter("vpc-id").withValues(group.getVpcId()));
                      filters.add(new Filter("state").withValues("available"));
                      filters.add(new Filter("subnet-id").withValues(getCurrentSubnetId()));
          
                      DescribeSubnetsRequest subnetReq = new DescribeSubnetsRequest();
                      subnetReq.withFilters(filters);
                      DescribeSubnetsResult subnetResult = ec2.describeSubnets(subnetReq);
          
                      List<Subnet> subnets = subnetResult.getSubnets();
                      if (subnets != null && !subnets.isEmpty()) {
                          groupIds.add(group.getGroupId());
                      }
                  }
              }
          
              if (securityGroupSet.size() != groupIds.size()) {
                  throw new AmazonClientException("Security groups must all be VPC security groups to work in a VPC context");
              }
          
              return groupIds;
          }
          
          davidfluck David Fluck added a comment - It looks like this change has indeed been released. I've also encountered that error before. How are you listing the subnets? Are you using subnet IDs? Are all of the subnets in the same VPC? This is the relevant code: private List< String > getEc2SecurityGroups(AmazonEC2 ec2) throws AmazonClientException { List< String > groupIds = new ArrayList< String >(); DescribeSecurityGroupsResult groupResult = getSecurityGroupsBy( "group-name" , securityGroupSet, ec2); if (groupResult.getSecurityGroups().size() == 0) { groupResult = getSecurityGroupsBy( "group-id" , securityGroupSet, ec2); } for (SecurityGroup group : groupResult.getSecurityGroups()) { if (group.getVpcId() != null && !group.getVpcId().isEmpty()) { List<Filter> filters = new ArrayList<Filter>(); filters.add( new Filter( "vpc-id" ).withValues(group.getVpcId())); filters.add( new Filter( "state" ).withValues( "available" )); filters.add( new Filter( "subnet-id" ).withValues(getCurrentSubnetId())); DescribeSubnetsRequest subnetReq = new DescribeSubnetsRequest(); subnetReq.withFilters(filters); DescribeSubnetsResult subnetResult = ec2.describeSubnets(subnetReq); List<Subnet> subnets = subnetResult.getSubnets(); if (subnets != null && !subnets.isEmpty()) { groupIds.add(group.getGroupId()); } } } if (securityGroupSet.size() != groupIds.size()) { throw new AmazonClientException( "Security groups must all be VPC security groups to work in a VPC context" ); } return groupIds; }
          nforbes Nicola Forbes added a comment -

          Hi David,

          Thanks for replying so quickly. I'm supplying the subnet ids as a comma-separated list (subnet-<string1>, subnet-<string2>, subnet-<string3>). All of the subnets are part of the same VPC but belong to different AZs.

          subnet-<string1> is part of AZ us-east-1a
          subnet-<string2> is part of AZ us-east-1b
          subnet-<string3> is part of AZ us-east-1c

          I am also listing the three AZs as a comma-separated list in the "availability zone" text box ie. us-east-1a, us-east-1b, us-east-1c

          It is possible that I have misunderstood the new feature. Do the subnets have to be part of the same availability zone?

          nforbes Nicola Forbes added a comment - Hi David, Thanks for replying so quickly. I'm supplying the subnet ids as a comma-separated list (subnet-<string1>, subnet-<string2>, subnet-<string3>). All of the subnets are part of the same VPC but belong to different AZs. subnet-<string1> is part of AZ us-east-1a subnet-<string2> is part of AZ us-east-1b subnet-<string3> is part of AZ us-east-1c I am also listing the three AZs as a comma-separated list in the "availability zone" text box ie. us-east-1a, us-east-1b, us-east-1c It is possible that I have misunderstood the new feature. Do the subnets have to be part of the same availability zone?
          equick Edward Quick added a comment -

          Hi Nicola, I could be wrong but a colleague of mine mentioned that you need to provide the AZ list as space delimited (not comma delimited).

          Ed

           

          equick Edward Quick added a comment - Hi Nicola, I could be wrong but a colleague of mine mentioned that you need to provide the AZ list as space delimited (not comma delimited). Ed  
          nforbes Nicola Forbes added a comment -

          Hi Ed,

          Thanks for the suggestion. I've just tried it out but unfortunately still get the same error (various combos tried). I haven't had a chance to read through the source code, but it looks like az can only take one availability zone, not a list, and so all subnets must be part of the same az.

          Nicky

          nforbes Nicola Forbes added a comment - Hi Ed, Thanks for the suggestion. I've just tried it out but unfortunately still get the same error (various combos tried). I haven't had a chance to read through the source code, but it looks like az can only take one availability zone, not a list, and so all subnets must be part of the same az. Nicky

          People

            francisu Francis Upton
            brockoli Brock Ackley
            Votes:
            3 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: