At the end of a failing build, the hudson.tasks.MailSender.buildCulpritList determines who to email.
The problem comes when hudson.scm.SubversionMailAddressResolverImpl.findMailAddressFor determines the email address of the user by finding all builds a user has committed to. This is done by iterating over every single Jenkins project (hudson.model.User.getProjects() first finds all projects and then uses AbstractProject.hasParticipant - which reads the changelog to see if the user participated).

For a large system (we have tens of thousands of builds), this is not at all efficient.

Unfortunately findMailAddressFor takes a user and not a project (as the obvious implementation would be to work out the email address from the commit).

Also, the results aren't cached and so this is run for every user every time.

I'm not sure if this can be resolved with just a fix to the subversion-plugin