While the openstack spec says that timestamps should be ISO 8601 format, not all OpenStack deployments do so. e.g. the one I have issues timestamps like "expires_at": "2017-08-30T13:46:51.480655Z"

The Openstack plugin version 2.24 uses openstack4j version 3.0.3 which uses jackson version 2.7.3 in a manner which results in "2017-08-30T13:46:51.480655Z" being treated as "2017-08-30T13:46:51Z plus 480655 milliseconds" rather than "2017-08-30T13:46:51Z plus 481 milliseconds". This results in the resulting java.util.Date coming back with the wrong data (8 minutes "fast" in this case).

In addition, even after an Openstack instance's isExpired() method returns true, JCloudsCloud.java caches the instance so that the code continues to use the expired session even after Openstack.isExpired has started returning true.

Initially this fault is harmless (few areas of the code care about Date values supplied by Openstack), but I've found that if the plugin uses its login Token after it has expired then Bad Things happen. e.g. Jenkins ceases to be able to spin up new slaves as the time thread responsible for spawning new slaves then deadlocks.

Ideally we would get the timestamp-parsing bug fixed in OpenStack4J and then upgrade the version we're using to the bugfixed one (this may in turn require a fix in Jackson and then for OpenStack4J to bump the version it uses). However, that will take a while to arrange (openstack4j issue 928 is a start, but is only a partial fix).

Until then, the openstack-cloud plugin needs a workaround, e.g. to treat Tokens as if they expire 999000 milliseconds (16m39s) earlier than the server actually said they did (and cope with potential mismatches between server time and Jenkins time, and allow for long-running operations), and to make sure that we don't use expired sessions at all.

i.e. the code needs to "expire" the Openstack instance early enough to prevent usage past its expiry time and needs to leave the caching to the Openstack class so that it doesn't use "expired" instances.