Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-66901

Add support for EC2 placement HostResourceGroupArn -parameter

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • ec2-plugin
    • None

      Please add support for HostResourceGroupArn in the Placement object. This would allow Jenkins EC2 plugin to provision an EC2 Mac instance on to a specific Host Resource Group. If this Host Resource Group doesn't have a free EC2 Mac dedicated host it will actually allocate one automatically. It will also release that EC2 Mac host after 24 hours if there isn't any Mac instances running on it.

      So by adding support for this parameter would allow EC2 Plugin to dynamically launch EC2 Macs without the need to have a pre provisioned fleet of unused EC2 Macs.

      Host Resource Groups is a feature of AWS License Manager: https://docs.aws.amazon.com/license-manager/latest/userguide/host-resource-groups.html
      You can use it without the need of having specific licenses to manage a group of dedicated hosts in the case of EC2 Macs. It supports automatic allocation and release of EC2 Macs.

          [JENKINS-66901] Add support for EC2 placement HostResourceGroupArn -parameter

          I have implemented support for host resource group ARN for Jenkins EC2 plugin and I'm going to submit the contribution next.

          I have successfully tested launching Mac instance using version with the feature and dedicated host was automatically allocated.

          I implemented the smallest possible change and only added host resource group parameter. To use this to launch a Mac instance, the license configuration is also required, but it can be associated with the AMI so it does not need to be set when launching the instance. I believe this is sufficient.

          Joonas Kuorilehto added a comment - I have implemented support for host resource group ARN for Jenkins EC2 plugin and I'm going to submit the contribution next. I have successfully tested launching Mac instance using version with the feature and dedicated host was automatically allocated. I implemented the smallest possible change and only added host resource group parameter. To use this to launch a Mac instance, the license configuration is also required, but it can be associated with the AMI so it does not need to be set when launching the instance. I believe this is sufficient.

          Joonas Kuorilehto added a comment - Pull request created https://github.com/jenkinsci/ec2-plugin/pull/784

          Turns out that the pull request is not needed to use Mac instances. Why?

          To use host resource group to automatically allocate EC2 mac dedicated hosts, you need both a Host Resource Group and a self managed license in AWS License Manager. When you specify the license configuration in the host resource group, it is SUFFICIENT to only specify the license configuration and the instance will be launched into the host resource group. The specifying of license configuration can be built into the AMI so no EC2 plugin change is required.

          So steps to use this:

          1. Create self managed license in AWS License Manager https://us-east-1.console.aws.amazon.com/license-manager/home?region=us-east-1#/licenseConfigurations . Choose licence type: Core. Per instance licenses won't work with host resource group host allocation.
          2. Create a Host Resource Group. I'll provide CloudFormation example.
          3. Associate the license with an AMI. You can build AMI with packer.

          When the license is associated with AMI and host resource group references the license configuration like this, it's enough to specify tenancy = Host and the dedicated host will be allocated and released automatically.

          Unable to find source-code formatter for language: shell. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
           
          aws --region REGION license-manager update-license-specifications-for-resource \
            --resource-arn "arn:aws:ec2:REGION:AMI_OWNER:image/$AMI_ID" \
            --add-license-specifications LicenseConfigurationArn=LICENSE_CONFIGURATION_ARN
          
            Mac1HostResourceGroup: 
              Type: AWS::ResourceGroups::Group
              Properties: 
                Name: mac-host-resource-group
                Configuration: 
                  - Type: AWS::EC2::HostManagement
                    Parameters: 
                      - Name: allowed-host-based-license-configurations
                        Values: 
                          - arn:aws:license-manager:us-east-1:123456789012:license-configuration:lic-4001aad2c09b8dbdd221eb59f5eFIXME
                      - Name: allowed-host-families
                        Values: 
                          - mac1
                          - mac2
                      - Name: auto-allocate-host
                        Values: 
                          - 'true'
                      - Name: auto-release-host
                        Values: 
                          - 'true'
                  - Type: AWS::ResourceGroups::Generic
                    Parameters: 
                      - Name: allowed-resource-types
                        Values: 
                          - AWS::EC2::Host
                      - Name: deletion-protection
                        Values: 
                          - UNLESS_EMPTY
          

          Or check out https://github.com/aws-samples/amazon-autoscaling-mac1metal-ec2-with-terraform

          In Jenkins EC2 plugin configuration, these are the relevant settings to get right (as JCASC):

                - description: "macos-12-x86_64_mac"
                  labelString: "macos"
                  amiFilters: 
                    - name: "name"
                      values: "ami-name-be-here"
                    - name: "state"
                      values: "available"
                  amiOwners: "self"
                  type: Mac1Metal
                  tenancy: Host
                  amiType: 
                    macData: 
                      sshPort: "22"
                  remoteAdmin: "ec2-user"
          

          Since the original motivation was for this host allocation function, I would suggest this ticket is marked resolved won't fix as this is a bit niche and adding these two parameters is a bit excessive. However, I would like to see another new feature for launch templates so it would be possible to also do this that way, and not always add more parameters to the EC2 plugin.

          Joonas Kuorilehto added a comment - Turns out that the pull request is not needed to use Mac instances. Why? To use host resource group to automatically allocate EC2 mac dedicated hosts, you need both a Host Resource Group and a self managed license in AWS License Manager. When you specify the license configuration in the host resource group, it is SUFFICIENT to only specify the license configuration and the instance will be launched into the host resource group. The specifying of license configuration can be built into the AMI so no EC2 plugin change is required. So steps to use this: Create self managed license in AWS License Manager https://us-east-1.console.aws.amazon.com/license-manager/home?region=us-east-1#/licenseConfigurations . Choose licence type: Core. Per instance licenses won't work with host resource group host allocation. Create a Host Resource Group. I'll provide CloudFormation example. Associate the license with an AMI. You can build AMI with packer. When the license is associated with AMI and host resource group references the license configuration like this, it's enough to specify tenancy = Host and the dedicated host will be allocated and released automatically. Unable to find source-code formatter for language: shell. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml   aws --region REGION license-manager update-license-specifications- for -resource \ --resource-arn "arn:aws:ec2:REGION:AMI_OWNER:image/$AMI_ID" \ --add-license-specifications LicenseConfigurationArn=LICENSE_CONFIGURATION_ARN Mac 1 HostResourceGroup: Type: AWS::ResourceGroups::Group Properties: Name: mac-host-resource-group Configuration: - Type: AWS::EC 2 ::HostManagement Parameters: - Name: allowed-host-based-license-configurations Values: - arn:aws:license-manager:us-east- 1 : 123456789012 :license-configuration:lic- 4001 aad 2 c 09 b 8 dbdd 221 eb 59 f 5 eFIXME - Name: allowed-host-families Values: - mac 1 - mac 2 - Name: auto-allocate-host Values: - 'true' - Name: auto-release-host Values: - 'true' - Type: AWS::ResourceGroups::Generic Parameters: - Name: allowed-resource-types Values: - AWS::EC 2 ::Host - Name: deletion-protection Values: - UNLESS_EMPTY Or check out https://github.com/aws-samples/amazon-autoscaling-mac1metal-ec2-with-terraform In Jenkins EC2 plugin configuration, these are the relevant settings to get right (as JCASC): - description: "macos- 12 -x 86 _ 64 _mac" labelString: "macos" amiFilters: - name: "name" values: "ami-name-be-here" - name: "state" values: "available" amiOwners: "self" type: Mac 1 Metal tenancy: Host amiType: macData: sshPort: " 22 " remoteAdmin: "ec 2 -user" Since the original motivation was for this host allocation function, I would suggest this ticket is marked resolved won't fix as this is a bit niche and adding these two parameters is a bit excessive. However, I would like to see another new feature for launch templates so it would be possible to also do this that way, and not always add more parameters to the EC2 plugin.

            thoulen FABRIZIO MANFREDI
            syvanen Toni Syvänen
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: