When an instance is provisioned (or started), awaitInstanceSshAvailable() succeeds before SSH is really allowing normal users to log in. Trying to log in via SSH will f.e. report "System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).".
This will result in the subsequent agent launch to fail and the node will never be usable (unless the agent is later relaunched by manually clicking on it) and it will go on and try to provision the next node etc.

I worked around the problem by patching awaitInstanceSshAvailable() to also log in as the user that the agent will be launched with, which solves the above problem (patch is attached).