Combination of two issues here.

1. According to Docker's docs, running docker build --build-arg VARIABLE should be equivalent to docker build --build-arg VARIABLE=${VARIABLE}. However, the Docker Pipeline plugin throws an exception if this usage is passed to docker.build in a pipeline. (See attached Jenkinsfile and log).
2. As shown in the log, these arguments are passed to the Docker client and the image builds fine. Only after this has happened is the exception logged and the build is marked as failed, despite the successful Docker build.

A workaround would be to just use --build-arg SSH_KEY="${env.SSH_KEY}", but as this would result in the private key being printed to the build logs I'm not really happy with that approach.