Sorry yes, 2.11 (corrected). Got confused on my versions since we have been upgrading quite a few things.

I agree our network setup is a bit non-standard. It's this way so we have a clear distinction between an actual private network that we use to communicate with other servers, and a "public" network for all non-LAN access. It is less important for Jenkins but for production, it means we are not mixing any customer traffic with private support services and, thus, we can more easily trust the private network. The other reason it is like this is, in part, because we don't have enough IPv4 address to give every VM a public IP but want to avoid most of the trainwreck that is NATing to one network (the "public"). Our private cloud provider does not yet support IPv6 addresses directly to VMs. If we could do that, we would likely have a true public network (with actual public IPs assigned to VMs) and a true private network (with a private v4 and unique-local v6).

From the standpoint of JClouds, it's seeing two networks which look to be private since that's true in the specific sense. But in practice, the Jenkins master can only see these servers over the network we consider as actually private.

For some concrete numbers, we have two networks, "inside" and "outside" that both have private IPs (again, the NAT monster for IPv4). Sometimes Jenkins tries to SSH to the "outside" network when it should always SSH into the inside. I had thought I could beat this with cloud-init but wasn't really able to come up with much there. The networks should always be the same (eth0 = outside, eth1 = inside) though.

Finally, yes 75% of the time it seemed to pick the wrong IP. Although more recent test show it's more like 50/50. If Jenkins tried all IPs that would eventually fix it but it didn't seem like it was doing that (if it can't connect to the outside IP, it will continue to try the outside IP rather than trying the other IP).

Ok, so since this is your own setup, you do know the expected IP ranges in Advance, right? E.g. lets say "internal" is 192.168.0.0/24 and "external" is 10.1.1.0/24 and when you display a machine's Details, you see it's Addresses sometimes like

Private:
192.168.0.123 10.1.1.19
Public:
none

and sometimes like

Private:
10.1.1.19 192.168.0.123
Public:
none

Since Jenkins always chooses the first of multiple Addresses in the same class it would choose 192.168.1.0.123 in the first case and 10.1.1.19 in the second case.

Spec

1. For you, It would be sufficient, if you could specify the correct network by it's IP/prefix and Jenkins would then match that against the multiple Addresses of a newly created Node.
2. You do not use any initScript functionality and do use a preconfigured jenkins account on the individual nodes. (because that would be done by a different ssh from within jclouds - which I'm afraid can not be influenced from within the jclouds plugin)

Is that correct?

On last question:
Did you specify the two ID's of the networks in the advanced settings of the template or are those networks specified as defaults in your openstack installation?
I'm asking, because there might be a chance, that if the IDs are specified in the template, the order of the assigned IP addresses might be constant and in the same order as the specified network IDs.

Yup correct, we definitely know the network ranges involved. And yep specifying the network via IP/prefix would actually be rather elegant I think. Also correct we do not currently use initScript and have a preconfigured jenkins account. The slaves are built from a pre-made image that sets up as much as we can ahead of time so the tests don't have to stand up as much stuff.

On the last question, I believe the networks are not default (that's been one minor issue we've had outside of Jenkins actually) - we always have to specify the networks we want (and of note, though not relevant to Jenkins, but some VMs in our OpenStack environment have other networks than the ones mentioned - mostly this is for load balancing of services via a hardware LB).

I had hoped the order of the networks would determine which IP address was to be tried first, but the results seem sort inconsistent there. We thought maybe accessIPv4 might be handing out the wrong value, though oddly in my tests so far, the actual value there for the VMs I've looked at is blank. If jClouds can look at that variable, though, I could see about figuring out how to make that default to our internal network.

Hope all that helps! Thanks for all the questions and for taking a look at the issue (doubly true for looking at it so quickly!)

Code changed in jenkins
User: Fritz Elfert
Path:
jclouds-plugin/pom.xml
jclouds-plugin/src/main/java/jenkins/plugins/jclouds/compute/JCloudsBuildWrapper.java
jclouds-plugin/src/main/java/jenkins/plugins/jclouds/compute/JCloudsLauncher.java
jclouds-plugin/src/main/java/jenkins/plugins/jclouds/compute/JCloudsSlave.java
jclouds-plugin/src/main/java/jenkins/plugins/jclouds/compute/JCloudsSlaveTemplate.java
jclouds-plugin/src/main/resources/jenkins/plugins/jclouds/compute/JCloudsSlaveTemplate/config.jelly
jclouds-plugin/src/main/resources/jenkins/plugins/jclouds/compute/JCloudsSlaveTemplate/help-preferredAddress.html
jclouds-plugin/src/test/java/jenkins/plugins/jclouds/compute/JCloudsSlaveTemplateTest.java
http://jenkins-ci.org/commit/jclouds-plugin/fc590e17533d693e9b7727336c744f280c832213
Log:
Fixes JENKINS-40342

Ok, I have something to test for you:
https://jenkins.ci.cloudbees.com/job/plugins/job/jclouds-plugin/org.jenkins-ci.plugins$jclouds-jenkins/lastSuccessfulBuild/artifact/org.jenkins-ci.plugins/jclouds-jenkins/2.12-SNAPSHOT/jclouds-jenkins-2.12-SNAPSHOT.hpi

This adds a new Textfield "Preferred Address" in the Advanced Settings of the template. There you can enter an address/prefix and that should then match one of the node's IP-Addresses.
Please report back if that fits your needs (or with any further suggestions). If everything is ok for you, I then will release v 2.12 with that change.

Cheers
-Fritz

Edit: URL changed to latest artifact.

Installed, thanks! I'm validating things right now. Having an issue I think that is more with Jenkins than the plugin so I'm working on that. Mostly I just wanted to update the issue so you knew I was taking a look.

In my testing so far, it seems to be working as described. I tried it with both networks just to make sure it wasn't do anything funky. I think our actual jobs are doing something odd I need to look at (and/or I picked the wrong job to use as a test), but the plugin itself seems to work as advertised (I see in the logs it's trying to connect where I tell it when it matches and likewise the UI has the proper IP bolded).

I need to test more, but so far, it looks like we're in business!

I just released v2.12 of the plugin.
So if you are ok with that, you can close the issue.

2.12 appeas to be working great! Thanks so much for getting this fixed up and in rapid order too!