The only way to make the agent connect to a Jenkins master with a self-signed HTTPS certificate seems to be by adding the certificate in the host. This is a problem with the Docker image because of the need to extend it.

The options -cert and -noCertificateCheck is already present in the remoting library, but not available for use with the -webSocket.