Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62939

Launching EC2 slaves is taking a very long time

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • ec2-plugin
    • None
    • Operating System: AWS EC2, Amazon Linux and Amazon Linux 2
      JRE/JDK: 8
      Jenkins version: 2.235.1 (LTS)
      Amazon EC2 plugin version: 1.50.3
      Amazon Web Services SDK version: 1.11.799

      We have been using the Amazon EC2 plugin for quite some time. In the past it would launch EC2 slaves very quickly. Now, it is taking a very long time. It appears to take over 7 minutes for each slave.

       

      We have lots of logs that look like:

      INFO: The instance EC2 (us-east-1) - Java 11 Slave (i-xyz) has a blank console. Maybe the console is yet not available. If enough time has passed, consider changing the key verification strategy or the AMI used by one printing out the host key in the instance console

      This seems to take long for both check-new-hard and check-new-soft slaves.

       

      I'm not sure how the EC2 plugin checks this. But, I did open the EC2 web console and performed "Actions->Instance Settings->Get System Log"

      I see lines like this a full 5 minutes before the EC2 plugin can verify the keys and connect:

       

      {{<14>Jul 2 21:09:05 ec2: }}
      <14>Jul 2 21:09:05 ec2: #############################################################
      <14>Jul 2 21:09:05 ec2: ----BEGIN SSH HOST KEY FINGERPRINTS----
      <14>Jul 2 21:09:05 ec2: 256 SHA256:XYZ no comment (ECDSA)
      <14>Jul 2 21:09:05 ec2: 256 SHA256:XYZ no comment (ED25519)
      <14>Jul 2 21:09:05 ec2: 2048 SHA256:XYZ no comment (RSA)
      <14>Jul 2 21:09:05 ec2: ----END SSH HOST KEY FINGERPRINTS----
      <14>Jul 2 21:09:05 ec2: #############################################################
      ----BEGIN SSH HOST KEY KEYS----
      ecdsa-sha2-nistp256 XYZ
      ssh-ed25519 XYZ
      ssh-rsa XYZ

       

      To be quite clear, the slaves do eventually connect. That is quite consistent. The issue here is the length of time has increased greatly.

            thoulen FABRIZIO MANFREDI
            dvenable David Venable
            Votes:
            7 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated: