Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45336

EC2 Slave Plugin launch instance and established slave agent connection with master and then dies

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • ec2-plugin
    • None

      Sees agent successfully connected online, but after few minutes it drops the connection throws the following error, 

      Jul 06, 2017 10:55:23 AM null FINEST: Node Jenkins Slave (i-0ee5cb818f57041df)(i-0ee5cb818f57041df) is still pending/launching, waiting 5s Jul 06, 2017 10:55:28 AM null FINEST: Node Jenkins Slave (i-0ee5cb818f57041df)(i-0ee5cb818f57041df) is still pending/launching, waiting 5s Jul 06, 2017 10:55:33 AM null FINER: Node Jenkins Slave (i-0ee5cb818f57041df)(i-0ee5cb818f57041df) is ready Jul 06, 2017 10:55:33 AM null INFO: Launching instance: i-0ee5cb818f57041df Jul 06, 2017 10:55:33 AM null INFO: bootstrap() Jul 06, 2017 10:55:33 AM null INFO: Getting keypair... Jul 06, 2017 10:55:33 AM null INFO: Using key: platforms-zoo 9c:3f:e9:56:e2:db:4d:34:5a:d8:d3:3c:d4:1a:35:d0 -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAtR+T7SGipA8DMsZ0dHkTrFcHbjgw+6o8sxkhmIQR4Z9SFXpZ2O6D1zREESwR L32waEVT1VQttXLTB9wUuKO6gf6YwVIEz4qfSvaXraxZpFk9RoE Jul 06, 2017 10:55:33 AM null INFO: Authenticating as ec2-user Jul 06, 2017 10:55:34 AM null INFO: Connecting to 10.26.22.216 on port 22, with timeout 10000. Jul 06, 2017 10:55:44 AM null INFO: Failed to connect via ssh: The kexTimeout (10000 ms) expired. Jul 06, 2017 10:55:44 AM null INFO: Waiting for SSH to come up. Sleeping 5. Jul 06, 2017 10:55:49 AM null INFO: Connecting to 10.26.22.216 on port 22, with timeout 10000. Jul 06, 2017 10:55:49 AM null INFO: Failed to connect via ssh: There was a problem while connecting to 10.26.22.216:22 Jul 06, 2017 10:55:49 AM null INFO: Waiting for SSH to come up. Sleeping 5. Jul 06, 2017 10:55:54 AM null INFO: Connecting to 10.26.22.216 on port 22, with timeout 10000. Jul 06, 2017 10:55:54 AM null INFO: Connected via SSH. Jul 06, 2017 10:55:54 AM null INFO: connect fresh as root Jul 06, 2017 10:55:54 AM null INFO: Connecting to 10.26.22.216 on port 22, with timeout 10000. Jul 06, 2017 10:55:54 AM null INFO: Connected via SSH. Jul 06, 2017 10:55:54 AM null INFO: Creating tmp directory (/tmp) if it does not exist Jul 06, 2017 10:55:54 AM null INFO: Executing init script Loaded plugins: priorities, update-motd, upgrade-helper No packages marked for update Jul 06, 2017 10:55:55 AM null INFO: Verifying that java exists openjdk full version "1.8.0_131-b11" Jul 06, 2017 10:55:57 AM null INFO: Copying slave.jar Jul 06, 2017 10:55:57 AM null INFO: Launching slave agent (via Trilead SSH2 Connection): java -jar /tmp/slave.jar <===[JENKINS REMOTING CAPACITY]===>Slave.jar version: 3.7 This is a Unix agent Evacuated stdout Agent successfully connected and online ERROR: Connection terminated java.io.EOFException at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2353) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2822) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:804) at java.io.ObjectInputStream.<init>(ObjectInputStream.java:301) at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59) Caused: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)

      I also see the following messages in ec2 systemlogs, soon after which the instance fails.. 

      Amazon Linux AMI release 2017.03
      Kernel 4.9.32-15.41.amzn1.x86_64 on an x86_64
      
      ip-10-26-22-216 login: /dev/fd/11: line 1: /sbin/plymouthd: No such file or directory
      initctl: Event failed
      Stopping atd: [  OK  ]
      
      Stopping sshd:  [  OK  ]
      
      Shutting down sm-client: [  OK  ]
      
      Shutting down sendmail: [  OK  ]
      
      Stopping crond: [  OK  ]
      
      Stopping acpi daemon: [  OK  ]
      
      Shutting down ntpd: [  OK  ]
      
      Stopping block device availability: Deactivating block devices:
      [  OK  ]
      
      Stopping system message bus: [  OK  ]
      
      Stopping rpcbind: [  OK  ]
      
      Stopping auditd: [  134.890516] audit: type=1305 audit(1499338670.301:103): audit_pid=0 old=2369 auid=4294967295 ses=4294967295 res=1
      [  OK  ]
      
      [  134.986560] audit: type=1305 audit(1499338670.397:104): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
      Shutting down system logger: [  OK  ]
      
      Shutting down interface eth0:  [  OK  ]
      
      Shutting down loopback interface:  [  OK  ]
      
      Stopping rngd: [  OK  ]
      
      Sending all processes the TERM signal... [  OK  ]
      
      Sending all processes the KILL signal... [  OK  ]
      
      Saving random seed:  [  OK  ]
      
      Turning off quotas:  [  OK  ]
      
      init: Re-executing /sbin/init
      
      [  140.696276] EXT4-fs (xvda1): re-mounted. Opts: (null)
      Halting system...
      [  140.962487] xenbus: xenbus_dev_shutdown: device/vfb/0: Initialising != Connected, skipping
      [  140.969041] ACPI: Preparing to enter system sleep state S5
      [  140.972889] reboot: Power down
      [  140.977963] acpi_power_off called

          [JENKINS-45336] EC2 Slave Plugin launch instance and established slave agent connection with master and then dies

          There are no comments yet on this issue.

            francisu Francis Upton
            mohammedn Nayeem Mohammed
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: