We just wanted to start using the workflow (sorry, pipeline) global lib feature.
When trying to clone the repository via ssh, it fails:

fatal: could not fetch refs from ssh://<user>@<server>:<port>/workflowLibs.git

Workaround is to clone via http, then add an "ssh remote" and push the initial commit. Cloning via ssh works once there is something in the repository.