Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16342

asynchPeople very slow when using Gravatar & Subversion plugins

    XMLWordPrintable

Details

    Description

      https://gist.github.com/4534378 shows https://jenkins.ci.cloudbees.com/job/libraries/asynchPeople/ taking an inordinate amount of time in the callback. Clearly AsynchPeople.data() cannot assume that UserAvatarResolver.resolve is quick. UserGravatarResolver sometimes blocks on an HTTP connection to a web service, though this is cached. However every time it checks Mailer.UserProperty.getAddress, which in the presence of SubversionMailAddressResolverImpl examines every build of every project in every folder in all of Jenkins. This is so inefficient we probably never want to do it, much less synchronously in an HTTP request.

      Besides People, the avatar is used in a couple of per-User pages, which is perhaps OK; but shown for every User in the HudsonPrivateSecurityRealm overview, which is definitely not OK.

      Attachments

        Issue Links

          Activity

            jglick Jesse Glick added a comment -

            Unreleased fixes of JENKINS-15440 at least avoid the problem when you have no configured rules. But if you have even one, the problem will suddenly pop up again.

            jglick Jesse Glick added a comment - Unreleased fixes of JENKINS-15440 at least avoid the problem when you have no configured rules. But if you have even one, the problem will suddenly pop up again.

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/model/View.java
            http://jenkins-ci.org/commit/jenkins/c757e65431ad1aec9a3bebe81ac919577e51ac58
            Log:
            JENKINS-16342 Improving responsiveness of asynchPeople when Gravatar plugin is in use.
            This change does not necessarily improve total performance, since the avatar is still computed.
            But (1) the computation is correctly done in the work thread, not in the HTTP response thread;
            (2) the computation is done just once for a given User, which could reduce load when many AJAX checks are done.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/model/View.java http://jenkins-ci.org/commit/jenkins/c757e65431ad1aec9a3bebe81ac919577e51ac58 Log: JENKINS-16342 Improving responsiveness of asynchPeople when Gravatar plugin is in use. This change does not necessarily improve total performance, since the avatar is still computed. But (1) the computation is correctly done in the work thread, not in the HTTP response thread; (2) the computation is done just once for a given User, which could reduce load when many AJAX checks are done.
            jglick Jesse Glick added a comment -

            Some things remain to be done to make UserAvatarResolver consistently quick:

            • SubversionMailAddressResolverImpl should probably be rewritten to perform its calculations in a long-running process and cache them. Or maybe this should just be deleted and the problem solved in a different way.
            • UserGravatarResolver should be rewritten to never block the API call waiting on an HTTP response; should initiate these asynchronously.

            And until then, or even afterwards in case other plugins will reintroduce similar bugs:

            • May be better for asynchPeople to list all users in a first pass, then go back and resolve avatars in a second pass.
            • HudsonPrivateSecurityRealm/index.jelly and similar pages should avoid asking for the avatar synchronously in the HTTP handling thread. Can use ProgressiveRendering or perform the avatar calculation in the URL handler for the <img> rather than when producing the page.
            jglick Jesse Glick added a comment - Some things remain to be done to make UserAvatarResolver consistently quick: SubversionMailAddressResolverImpl should probably be rewritten to perform its calculations in a long-running process and cache them. Or maybe this should just be deleted and the problem solved in a different way. UserGravatarResolver should be rewritten to never block the API call waiting on an HTTP response; should initiate these asynchronously. And until then, or even afterwards in case other plugins will reintroduce similar bugs: May be better for asynchPeople to list all users in a first pass, then go back and resolve avatars in a second pass. HudsonPrivateSecurityRealm/index.jelly and similar pages should avoid asking for the avatar synchronously in the HTTP handling thread. Can use ProgressiveRendering or perform the avatar calculation in the URL handler for the <img> rather than when producing the page.
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #2191
            JENKINS-16342 Improving responsiveness of asynchPeople when Gravatar plugin is in use. (Revision c757e65431ad1aec9a3bebe81ac919577e51ac58)

            Result = SUCCESS
            Jesse Glick : c757e65431ad1aec9a3bebe81ac919577e51ac58
            Files :

            • core/src/main/java/hudson/model/View.java
            • changelog.html
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2191 JENKINS-16342 Improving responsiveness of asynchPeople when Gravatar plugin is in use. (Revision c757e65431ad1aec9a3bebe81ac919577e51ac58) Result = SUCCESS Jesse Glick : c757e65431ad1aec9a3bebe81ac919577e51ac58 Files : core/src/main/java/hudson/model/View.java changelog.html
            jglick Jesse Glick added a comment -

            Serious enough to potentially merit backport to 1.480.x.

            jglick Jesse Glick added a comment - Serious enough to potentially merit backport to 1.480.x.
            jglick Jesse Glick added a comment -

            Gravatar plugin could probably take advantage of new resolveFast.

            jglick Jesse Glick added a comment - Gravatar plugin could probably take advantage of new resolveFast .
            kutzi kutzi added a comment -

            > SubversionMailAddressResolverImpl should probably be rewritten to perform its calculations in a long-running process and cache them. Or maybe this should just be deleted and the problem solved in a different way.

            Absolutely +1 for deleting this - as well as in all other SCM plugin where something analogue is implemented!
            These MailAddressResolvers are implemented in a so ridiculous inefficient way and their usefulness is IMHO very very limited.

            kutzi kutzi added a comment - > SubversionMailAddressResolverImpl should probably be rewritten to perform its calculations in a long-running process and cache them. Or maybe this should just be deleted and the problem solved in a different way. Absolutely +1 for deleting this - as well as in all other SCM plugin where something analogue is implemented! These MailAddressResolvers are implemented in a so ridiculous inefficient way and their usefulness is IMHO very very limited.
            vjuranek vjuranek added a comment -

            +1 to remove MailAddressResolvers or at least make them optional

            vjuranek vjuranek added a comment - +1 to remove MailAddressResolvers or at least make them optional
            jglick Jesse Glick added a comment -

            @kutzi, @vjuranek: see JENKINS-16437.

            jglick Jesse Glick added a comment - @kutzi, @vjuranek: see JENKINS-16437 .

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/model/View.java
            http://jenkins-ci.org/commit/jenkins/bd03abbcc0078c2f250c2e3ee2c8d3c4a9d210ca
            Log:
            JENKINS-16342 Improving responsiveness of asynchPeople when Gravatar plugin is in use.
            This change does not necessarily improve total performance, since the avatar is still computed.
            But (1) the computation is correctly done in the work thread, not in the HTTP response thread;
            (2) the computation is done just once for a given User, which could reduce load when many AJAX checks are done.(cherry picked from commit c757e65431ad1aec9a3bebe81ac919577e51ac58)

            Conflicts:
            changelog.html

            Compare: https://github.com/jenkinsci/jenkins/compare/cb6f200e8663...bd03abbcc007

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/model/View.java http://jenkins-ci.org/commit/jenkins/bd03abbcc0078c2f250c2e3ee2c8d3c4a9d210ca Log: JENKINS-16342 Improving responsiveness of asynchPeople when Gravatar plugin is in use. This change does not necessarily improve total performance, since the avatar is still computed. But (1) the computation is correctly done in the work thread, not in the HTTP response thread; (2) the computation is done just once for a given User, which could reduce load when many AJAX checks are done.(cherry picked from commit c757e65431ad1aec9a3bebe81ac919577e51ac58) Conflicts: changelog.html Compare: https://github.com/jenkinsci/jenkins/compare/cb6f200e8663...bd03abbcc007
            kutzi kutzi added a comment -

            Now that JENKINS-15440 is fixed, I think this can be closed, too.

            kutzi kutzi added a comment - Now that JENKINS-15440 is fixed, I think this can be closed, too.
            jglick Jesse Glick added a comment -

            @kutzi I think there is still work to be done in making UserGravatarResolver nonblocking (independently of MailAddressResolver) and/or making various UI elements displaying avatars not block rendering on the result. But these things do not need to be tracked in this issue I suppose.

            jglick Jesse Glick added a comment - @kutzi I think there is still work to be done in making UserGravatarResolver nonblocking (independently of MailAddressResolver ) and/or making various UI elements displaying avatars not block rendering on the result. But these things do not need to be tracked in this issue I suppose.

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: