Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19943

EC2 plugin cannot connect to slaves outside us-east-1 on new AWS accounts (using EC2-VPC)

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • ec2-plugin
    • None
    • Jenkins 1.534, plugin version 1.18, running on Ubuntu 12.10

    Description

      When running jenkins on a traditional host and configuring to launch EC2 slaves with a new AWS account in any region other than us-east-1, the plugin successfully creates an instance but fails to connect or report the instance online.

      The slave log shows the plugin is stuck attempting to SSH to the instance private IP, which is unreachable outside EC2. The node configure page correctly shows both public DNS and private IP.

      The issue appears to be because new AWS accounts are automatically being migrated to use EC2-VPC. On new accounts, all instances in regions other than us-east-1 are launched in a "default VPC". The default VPC combines behaviour from EC2-Classic and the original VPC implementation. In particular, although instances report they are part of a VPC they are still allocated a public DNS and public IP reachable from external hosts.

      The plugin code in the EC2UnixLauncher assumes that any instance with a VPC ID does not have a public DNS and falls back to using the private IP for connection. This appears to be the cause of this issue.

      As far as I can tell, there are now four potential cases:

      Jenkins Master Slave Default connection
      Internet EC2-Classic Slave public DNS
      Internet EC2-VPC with public IP Slave public DNS
      EC2-VPC EC2-VPC with public IP Slave public DNS(1)
      EC2-VPC EC2-VPC with no public IP Slave private IP

      (1) When a master in a VPC resolves public DNS for an instance in the same VPC, Amazon automatically returns the private IP.

      That suggests that the resolution should be to simplify the conditional in the EC2UnixLauncher and just connect to the public DNS if the instance has one, or the private IP if not.

      I don't currently have a java/jenkins plugin build environment available, so I can't easily test this myself.

      Attachments

        Issue Links

          Activity

            Still facing the issue here...running my master on Rackspace and slave on EC2. As a quick hacky work-around, I've checked the "Use private DNS" box in Manage Jenkins > Cloud > Amazon EC2 > Advanced...then mapped the private DNS name of my EC2 instance to the public IP address in master's /etc/hosts.

            rstanchak Roman Stanchak added a comment - Still facing the issue here...running my master on Rackspace and slave on EC2. As a quick hacky work-around, I've checked the "Use private DNS" box in Manage Jenkins > Cloud > Amazon EC2 > Advanced...then mapped the private DNS name of my EC2 instance to the public IP address in master's /etc/hosts.

            Code changed in jenkins
            User: Jens Brejner
            Path:
            src/main/java/hudson/plugins/ec2/ssh/EC2UnixLauncher.java
            http://jenkins-ci.org/commit/ec2-plugin/b469410a8245f05bfb4bbc96b243ac54e5bc1ecb
            Log:
            FIXED JENKINS-19943 but credit should go to Nathan Kinder

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jens Brejner Path: src/main/java/hudson/plugins/ec2/ssh/EC2UnixLauncher.java http://jenkins-ci.org/commit/ec2-plugin/b469410a8245f05bfb4bbc96b243ac54e5bc1ecb Log: FIXED JENKINS-19943 but credit should go to Nathan Kinder

            Code changed in jenkins
            User: Francis Upton
            Path:
            src/main/java/hudson/plugins/ec2/ssh/EC2UnixLauncher.java
            http://jenkins-ci.org/commit/ec2-plugin/f9b87476e96f322b19683743db4ee6bb5d07522c
            Log:
            Merge pull request #77 from jbrejner/master

            FIXED JENKINS-19943 but credit should go to Nathan Kinder

            Compare: https://github.com/jenkinsci/ec2-plugin/compare/6e994e07baf4...f9b87476e96f

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Francis Upton Path: src/main/java/hudson/plugins/ec2/ssh/EC2UnixLauncher.java http://jenkins-ci.org/commit/ec2-plugin/f9b87476e96f322b19683743db4ee6bb5d07522c Log: Merge pull request #77 from jbrejner/master FIXED JENKINS-19943 but credit should go to Nathan Kinder Compare: https://github.com/jenkinsci/ec2-plugin/compare/6e994e07baf4...f9b87476e96f

            Code changed in jenkins
            User: Steve
            Path:
            src/main/java/hudson/plugins/ec2/win/EC2WindowsLauncher.java
            http://jenkins-ci.org/commit/ec2-plugin/00cfb671b6f50bc28d84909927ed0b1da6776243
            Log:
            JENKINS-19943 Fix connection to Windows slaves outside us-east-1.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Steve Path: src/main/java/hudson/plugins/ec2/win/EC2WindowsLauncher.java http://jenkins-ci.org/commit/ec2-plugin/00cfb671b6f50bc28d84909927ed0b1da6776243 Log: JENKINS-19943 Fix connection to Windows slaves outside us-east-1.

            Code changed in jenkins
            User: Francis Upton
            Path:
            src/main/java/hudson/plugins/ec2/win/EC2WindowsLauncher.java
            http://jenkins-ci.org/commit/ec2-plugin/b99f8191e990a03d3f8c8a9887062f2ed8c34883
            Log:
            Merge pull request #110 from stevedenman/master

            JENKINS-19943 Fix connection to Windows slaves outside us-east-1.

            Compare: https://github.com/jenkinsci/ec2-plugin/compare/7fe8fdf29514...b99f8191e990

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Francis Upton Path: src/main/java/hudson/plugins/ec2/win/EC2WindowsLauncher.java http://jenkins-ci.org/commit/ec2-plugin/b99f8191e990a03d3f8c8a9887062f2ed8c34883 Log: Merge pull request #110 from stevedenman/master JENKINS-19943 Fix connection to Windows slaves outside us-east-1. Compare: https://github.com/jenkinsci/ec2-plugin/compare/7fe8fdf29514...b99f8191e990

            People

              francisu Francis Upton
              acoulton Andrew Coulton
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: