There are also related issues here when the hostname/port associated with Jenkins' URL (typically pointing to Apache) don't match those associated with the Jenkins container. For instance, I have Apache running on port 80 and Tomcat running on 8080, but I'd rather not 'publish' the 8080 address whenever Jenkins links to itself. I solved this issue in the interim by using an AJP connector between Tomcat and Apache so that the URLs will match, but it would be nice to have a solution that doesn't require me to alter my deployment.
Examining the code, it looks like the receivingURL is being pulled from the request. Changing this to the URL that was sent as returnTo (Hudson.getInstance().getRootUrl()+ finishUrl) would likely solve the problem. In my opinion the openid4java javadoc is a bit misleading here because you really want to verify that the returnTo matches what you sent, not where you happen to be deployed.