Jenkins slaves are being stopped multiple times with ec2 plugin

XMLWordPrintable

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Minor
    • Component/s: ec2-plugin
    • Environment:
      Jenkins Version: 2.60.3
      Amazon EC2 plugins: 1.36
      Idle termination time: -5

      I am seeing cases where Jenkins slaves are being stopped two or three times when using the ec2 plugin.  I was able to go to the AWS console and confirm the slave was successfully stopped after the first request. 

      Update: We changed our local branch to only go through the logic if the agent isOnline() which seemed to resolve the issue.

      From /var/log/jenkins/jenkins.log:

      Aug 26, 2017 8:02:05 AM hudson.plugins.ec2.EC2RetentionStrategy internalCheck
INFO: Idle timeout of jenkins-slave-name (i-xxxxxxxxxxxxxxxx) after 24 idle minutes, with 3 minutes remaining in billing period

Aug 26, 2017 8:02:05 AM hudson.plugins.ec2.EC2AbstractSlave idleTimeout
INFO: EC2 instance idle time expired: i-xxxxxxxxxxxxxxxx

Aug 26, 2017 8:02:06 AM hudson.plugins.ec2.EC2AbstractSlave stop
INFO: EC2 instance stop request sent for i-xxxxxxxxxxxxxxxx


Aug 26, 2017 8:03:59 AM hudson.plugins.ec2.EC2RetentionStrategy internalCheck
INFO: Idle timeout of jenkins-slave-name (i-xxxxxxxxxxxxxxxx) after 26 idle minutes, with 2 minutes remaining in billing period

Aug 26, 2017 8:03:59 AM hudson.plugins.ec2.EC2AbstractSlave idleTimeout
INFO: EC2 instance idle time expired: i-xxxxxxxxxxxxxxxx

Aug 26, 2017 8:04:00 AM hudson.plugins.ec2.EC2AbstractSlave stop
INFO: EC2 instance stop request sent for i-xxxxxxxxxxxxxxxx

       

      From AWS console:

      State transition reason:
User initiated (2017-08-26 08:02:05 GMT)
    
State transition reason message:
Client.UserInitiatedShutdown: User initiated shutdown

            Assignee:
            Francis Upton
            Reporter:
            Mike Kozell
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: