Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15440

Emailing users at the end of a failed build very slow for big Jenkins instance using subversion

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • subversion-plugin
    • Jenkins 1.447 LTS
      Subversion 1.35
      Although both Jenkins and subversion plugin are not latest version, I have browsed github for latest versions and I believe this issue to still be present.

      At the end of a failing build, the hudson.tasks.MailSender.buildCulpritList determines who to email.
      The problem comes when hudson.scm.SubversionMailAddressResolverImpl.findMailAddressFor determines the email address of the user by finding all builds a user has committed to. This is done by iterating over every single Jenkins project (hudson.model.User.getProjects() first finds all projects and then uses AbstractProject.hasParticipant - which reads the changelog to see if the user participated).

      For a large system (we have tens of thousands of builds), this is not at all efficient.

      Unfortunately findMailAddressFor takes a user and not a project (as the obvious implementation would be to work out the email address from the commit).

      Also, the results aren't cached and so this is run for every user every time.

      I'm not sure if this can be resolved with just a fix to the subversion-plugin

          [JENKINS-15440] Emailing users at the end of a failed build very slow for big Jenkins instance using subversion

          kutzi added a comment -

          Uh, these svn servers which are used by default in SubversionMailAddressResolverImpl.findMailAddressFor are very old.
          I'd suggest to remove them altogether even if that would mean a minor backword compatibility break - I cannot imagine that someone is really still relying on this feature.

          Also it would be an improvement to check first if there are any resolver rules and only if iterate over all jobs!

          kutzi added a comment - Uh, these svn servers which are used by default in SubversionMailAddressResolverImpl.findMailAddressFor are very old. I'd suggest to remove them altogether even if that would mean a minor backword compatibility break - I cannot imagine that someone is really still relying on this feature. Also it would be an improvement to check first if there are any resolver rules and only if iterate over all jobs!

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          src/main/java/hudson/scm/SubversionMailAddressResolverImpl.java
          http://jenkins-ci.org/commit/subversion-plugin/c7086b3d551e2bf4abc56fe841358b6e06258901
          Log:
          JENKINS-15440 stop as early as possible if there're no rules defined. Getting all projects of a user can be very expensive.

          Compare: https://github.com/jenkinsci/subversion-plugin/compare/9d8d24463bb9...c7086b3d551e

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/scm/SubversionMailAddressResolverImpl.java http://jenkins-ci.org/commit/subversion-plugin/c7086b3d551e2bf4abc56fe841358b6e06258901 Log: JENKINS-15440 stop as early as possible if there're no rules defined. Getting all projects of a user can be very expensive. Compare: https://github.com/jenkinsci/subversion-plugin/compare/9d8d24463bb9...c7086b3d551e

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          src/main/java/hudson/scm/SubversionMailAddressResolverImpl.java
          src/main/resources/hudson/scm/SubversionMailAddressResolverImpl/global.jelly
          http://jenkins-ci.org/commit/subversion-plugin/ad7882e97d026827a21d206e7731bfb61e746839
          Log:
          Removed very old legacy svn repositories which where configured as defaults for mail address resolving, but are very unlikely to be of use to anyone[FIXED JENKINS-15440]

          Compare: https://github.com/jenkinsci/subversion-plugin/compare/1795b677b3fd...ad7882e97d02

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/scm/SubversionMailAddressResolverImpl.java src/main/resources/hudson/scm/SubversionMailAddressResolverImpl/global.jelly http://jenkins-ci.org/commit/subversion-plugin/ad7882e97d026827a21d206e7731bfb61e746839 Log: Removed very old legacy svn repositories which where configured as defaults for mail address resolving, but are very unlikely to be of use to anyone [FIXED JENKINS-15440] Compare: https://github.com/jenkinsci/subversion-plugin/compare/1795b677b3fd...ad7882e97d02

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/resources/hudson/scm/SubversionMailAddressResolverImpl/global.jelly
          http://jenkins-ci.org/commit/subversion-plugin/d5f9fb5a4a30aeebb379c3214508ecf80c387382
          Log:
          JENKINS-15440 Warning about using mail resolver rules.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/resources/hudson/scm/SubversionMailAddressResolverImpl/global.jelly http://jenkins-ci.org/commit/subversion-plugin/d5f9fb5a4a30aeebb379c3214508ecf80c387382 Log: JENKINS-15440 Warning about using mail resolver rules.

          kutzi added a comment -

          Note that the same applies for the cvs-plugin which still - AFAIK - comes preinstalled with Jenkins.
          So just fixing this for svn doesn't help as long as cvs-plugin is enabled

          kutzi added a comment - Note that the same applies for the cvs-plugin which still - AFAIK - comes preinstalled with Jenkins. So just fixing this for svn doesn't help as long as cvs-plugin is enabled

          kutzi added a comment -

          This just completely killed our server, because this also seems to be triggered by requests to //job/JOBNAME/api/json
          I have to figure out where these request come from, but generally I think that this resolving 'feature' is after all a very bad idea.

          I'd really like to remove it altogether as I cannot really think of any scenario where this would have been actually useful (given the legacy state of the previously used hard-coded svn repositories)

          kutzi added a comment - This just completely killed our server, because this also seems to be triggered by requests to //job/JOBNAME/api/json I have to figure out where these request come from, but generally I think that this resolving 'feature' is after all a very bad idea. I'd really like to remove it altogether as I cannot really think of any scenario where this would have been actually useful (given the legacy state of the previously used hard-coded svn repositories)

          Jesse Glick added a comment -

          As mentioned in JENKINS-16342 I agree that this needs to be either rewritten or deleted outright.

          Jesse Glick added a comment - As mentioned in JENKINS-16342 I agree that this needs to be either rewritten or deleted outright.

          Code changed in jenkins
          User: Nicolas De Loof
          Path:
          src/main/java/hudson/scm/SubversionMailAddressResolverImpl.java
          http://jenkins-ci.org/commit/subversion-plugin/b2451c01790b19ab665a880857e641a46ed89828
          Log:
          JENKINS-15440 MailAddressResolverImpl moved to subversion-mail-address-resolver-plugin

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: src/main/java/hudson/scm/SubversionMailAddressResolverImpl.java http://jenkins-ci.org/commit/subversion-plugin/b2451c01790b19ab665a880857e641a46ed89828 Log: JENKINS-15440 MailAddressResolverImpl moved to subversion-mail-address-resolver-plugin

          MailAddressResolverImpl moved to https://github.com/jenkinsci/subversion-mail-address-resolver-plugin, just in case someone want to keep a 100% compatible jenkins. To be released after subversion 1.45

          Nicolas De Loof added a comment - MailAddressResolverImpl moved to https://github.com/jenkinsci/subversion-mail-address-resolver-plugin , just in case someone want to keep a 100% compatible jenkins. To be released after subversion 1.45

          kutzi added a comment -

          Sounds great. I'll push a svn-plugin release today or tomorrow.

          kutzi added a comment - Sounds great. I'll push a svn-plugin release today or tomorrow.

            kutzi kutzi
            willthames Will Thames
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: