-
Bug
-
Resolution: Fixed
-
Blocker
-
None
If multiple EC2 clouds are defined in the plugin, incorrect credentials seem to be used to issue commands for nodes in the second defined cloud. Instances for the second cloud can be created, but the Jenkins agent fails to connect, which results in a new node being created for each failed attempt. Once created, the nodes cannot be deleted until the first cloud is deleted.
[JENKINS-15081] Errors with multiple EC2 Clouds
Quentin Hartman
created issue -
Quentin Hartman
added a comment - Nope, the initial creation failure was not an SSH key problem. It seems that if you have multiple EC2 clouds configured, the settings for the first cloud in the list are what get used when instantiating or deleting instances, regardless of which cloud they actually belong to. The initial step of telling AWS to start a new instance works correctly, but all the steps after that fail because of credential problems.
Quentin Hartman
added a comment - Nope, the initial creation failure was not an SSH key problem. It seems that if you have multiple EC2 clouds configured, the settings for the first cloud in the list are what get used when instantiating or deleting instances, regardless of which cloud they actually belong to. The initial step of telling AWS to start a new instance works correctly, but all the steps after that fail because of credential problems. 
Ray Sennewald
added a comment - I'm seeing this same issue. This creates a problem for me because I have instances on different zones (us-east and us-west2 for example) but because of this am limited to only one zone per Jenkins server. Any idea on when we may have a fix for this?
Ray Sennewald
added a comment - I'm seeing this same issue. This creates a problem for me because I have instances on different zones (us-east and us-west2 for example) but because of this am limited to only one zone per Jenkins server. Any idea on when we may have a fix for this? Ray Sennewald
added a comment - I have a fix for this and I have a pull request open to get this pulled in to the main ec2-plugin repo. If your interested you can view the PR here: https://github.com/jenkinsci/ec2-plugin/pull/46 .
Basically what I found is previously the plugin was typically retrieving the first EC2Cloud instead of the EC2Cloud that the specific slave belongs to. This is what I've addressed in my changes. I tested it and confirmed with two different EC2Clouds added and two templates (one from each cloud), that I could properly add and delete instances from each template, which I previously couldn't do with any templates from the second cloud.
Ray Sennewald
added a comment - I have a fix for this and I have a pull request open to get this pulled in to the main ec2-plugin repo. If your interested you can view the PR here: https://github.com/jenkinsci/ec2-plugin/pull/46 .
Basically what I found is previously the plugin was typically retrieving the first EC2Cloud instead of the EC2Cloud that the specific slave belongs to. This is what I've addressed in my changes. I tested it and confirmed with two different EC2Clouds added and two templates (one from each cloud), that I could properly add and delete instances from each template, which I previously couldn't do with any templates from the second cloud. Quentin Hartman
added a comment - Awesome. I'll check that out as soon as I'm able. I'm kinda buried at the moment, so Francis might beat me to it.
Quentin Hartman
added a comment - Awesome. I'll check that out as soon as I'm able. I'm kinda buried at the moment, so Francis might beat me to it. Ray Sennewald
added a comment - Francis got to it and asked me to pull in his recent updates for spot instances. So I've closed my original PR, merged in his code, revised my modifications and issued a new PR: https://github.com/jenkinsci/ec2-plugin/pull/48 .
Ray Sennewald
added a comment - Francis got to it and asked me to pull in his recent updates for spot instances. So I've closed my original PR, merged in his code, revised my modifications and issued a new PR: https://github.com/jenkinsci/ec2-plugin/pull/48 . Code changed in jenkins
User: ray.sennewald@gmail.com
Path:
src/main/java/hudson/plugins/ec2/EC2AbstractSlave.java
src/main/java/hudson/plugins/ec2/EC2Computer.java
src/main/java/hudson/plugins/ec2/EC2ComputerLauncher.java
src/main/java/hudson/plugins/ec2/EC2OndemandSlave.java
src/main/java/hudson/plugins/ec2/EC2SpotSlave.java
src/main/java/hudson/plugins/ec2/SlaveTemplate.java
src/main/java/hudson/plugins/ec2/ssh/EC2UnixLauncher.java
http://jenkins-ci.org/commit/ec2-plugin/ec7e8e17979e60c4af631474611b1144a48b825f
Log:
Fix for JENKINS-15081. We store the EC2Cloud that a slave belongs to as an attribute in the EC2AbstractSlave class and have a function call to retrieve it in EC2Computer.
All references to EC2Cloud.get changed to EC2AbstractSlave.cloud or EC2Computer.getCloud().
This fix allows for us to always know the correct Cloud that an EC2AbstractSlave or EC2Computer belongs to so we can have multiple EC2Clouds configured on one Jenkins instance and have different templates for different clouds and have the instances for each spin up properly.
Code changed in jenkins
User: Francis Upton
Path:
src/main/java/hudson/plugins/ec2/EC2AbstractSlave.java
src/main/java/hudson/plugins/ec2/EC2Computer.java
src/main/java/hudson/plugins/ec2/EC2ComputerLauncher.java
src/main/java/hudson/plugins/ec2/EC2OndemandSlave.java
src/main/java/hudson/plugins/ec2/EC2SpotSlave.java
src/main/java/hudson/plugins/ec2/SlaveTemplate.java
src/main/java/hudson/plugins/ec2/ssh/EC2UnixLauncher.java
http://jenkins-ci.org/commit/ec2-plugin/5da8f6be66e839e33923cbf5e8b62b0f6c4b7bfc
Log:
Merge pull request #48 from rsennewald/jenkins-15081
Fix to allow multiple EC2Clouds (fix for JENKINS-15081)
Compare: https://github.com/jenkinsci/ec2-plugin/compare/881b2b80e488...5da8f6be66e8
Francis Upton
made changes -
| Resolution | New: Fixed [ 1 ] | |
| Status | Original: Open [ 1 ] | New: Resolved [ 5 ] |
The initial creation failure may be a result of not having the correct SSH key installed in the AMI. Investigating that now. The deletion problem is definitely real though.