Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54071

EC2-plugin not spooling up stopped nodes

    XMLWordPrintable

Details

    Description

      Stopped ec2 instances are no longer being resumed when needed.  Instead, new instances are spun up.   

      Instances were correctly resumed in version 1.39 but not in 1.40.1.   Reverting to 1.39 fixed the issue for me.

      This recent pull request appears to have removed the logic necessary to resume stopped instances.

      https://github.com/jenkinsci/ec2-plugin/pull/252

      Specifically the changes here: 

      https://github.com/jenkinsci/ec2-plugin/pull/252/files#diff-f2115e33148d3db7c133fe014ad9dfddR419

       

      Attachments

        Issue Links

          Activity

            It just waits for the servers to respond but it works fine after rolling back to the PR252 version.

            oliverp Oliver Pereira added a comment - It just waits for the servers to respond but it works fine after rolling back to the PR252 version.

            Could someone confirm, this was fixed in 1.41? I made sure, I added iam:ListInstanceProfilesForRole to the master's role policy. Then I terminated the old slave (only 1 currently). Building a job now correctly spins up a new instance (cap for this AMI is set to 1). Afterwards the instance is stopped after 1 minute of idling. Then for the next build it only seems to attempt to provision another machine (which of course fails with the cap limit). see the attached ec2.log

            tuky Tobias Krönke added a comment - Could someone confirm, this was fixed in 1.41? I made sure, I added iam:ListInstanceProfilesForRole to the master's role policy. Then I terminated the old slave (only 1 currently). Building a job now correctly spins up a new instance (cap for this AMI is set to 1). Afterwards the instance is stopped after 1 minute of idling. Then for the next build it only seems to attempt to provision another machine (which of course fails with the cap limit). see the attached  ec2.log
            paulbovbel Paul Bovbel added a comment -

            I believe I see the same issue as Tobias in 1.41

            paulbovbel Paul Bovbel added a comment - I believe I see the same issue as Tobias in 1.41
            thoulen FABRIZIO MANFREDI added a comment - - edited

            paulbovbel can you check if you didn't reach the cap ? there is a bug in the EC2 that is counting the stopped instances as running, that means when you reach the CAP is not able to restart the instance in the stop state (duplication of the JENKINS-53920)

             

            thoulen FABRIZIO MANFREDI added a comment - - edited paulbovbel can you check if you didn't reach the cap ? there is a bug in the EC2 that is counting the stopped instances as running, that means when you reach the CAP is not able to restart the instance in the stop state (duplication of the  JENKINS-53920 )  
            paulbovbel Paul Bovbel added a comment -

            That sounds about right Fabrizio, I'll keep an eye on https://issues.jenkins-ci.org/browse/JENKINS-53920.

            paulbovbel Paul Bovbel added a comment - That sounds about right Fabrizio, I'll keep an eye on  https://issues.jenkins-ci.org/browse/JENKINS-53920 .

            People

              thoulen FABRIZIO MANFREDI
              brycedrennan Bryce Drennan
              Votes:
              7 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: