Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20892

/{view,computer,user}/*/builds & /job/*/buildTimeTrend block HTTP response on build record loading

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Computer/builds.jelly calls Computer.getBuilds which first looks up all Job instances—probably not too expensive—then creates a merged RunList from them—again OK, as this is lazy—then calls the node(Node) filter. The filter is incremental, but if most builds were not on this computer, it would load many more than the limit of 50 in t:buildListTable before it finds some to display.

      Trimming the merged list before filtering would solve that problem (making this perform much like View/builds.jelly), but then you might not see anywhere near 50 builds, or indeed any at all, even when there are some (older) builds run on this computer.

      User/builds.jelly calls User.getBuilds which does not even bother using an incremental filter. It does use newBuilds to limit how many builds are considered per job, but then scans all of those at once for mentions of the current user.

      Ideally all such displays would show all applicable builds, but use ProgressiveRendering so that build records are only loaded in a background thread and only so long as you keep the page open. (Compare AsynchPeople.)

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment - - edited

            Actually it looks like even /view/*/builds is wrong: t:buildListTable truncates builds using Functions.subList, which to comply with the contract of List calls size, which on RunList forces all builds to be loaded!

            Show
            jglick Jesse Glick added a comment - - edited Actually it looks like even /view/*/builds is wrong: t:buildListTable truncates builds using Functions.subList , which to comply with the contract of List calls size , which on RunList forces all builds to be loaded!
            Hide
            jglick Jesse Glick added a comment -

            /view/*/builds and /computer/*/builds (but strangely enough not /user/*/builds) complicate the issue by using a BuildTimelineWidget, which seems incompatible with lazy loading.

            Show
            jglick Jesse Glick added a comment - /view/*/builds and /computer/*/builds (but strangely enough not /user/*/builds ) complicate the issue by using a BuildTimelineWidget , which seems incompatible with lazy loading.
            Hide
            jglick Jesse Glick added a comment -

            For User the RSS feeds are also bad; rssAll and rssFailed suffer from the same problem as builds. rssLatest is even worse: it will load every build of a job (not just newBuilds) unless and until it finds one associated with this user.

            Show
            jglick Jesse Glick added a comment - For User the RSS feeds are also bad; rssAll and rssFailed suffer from the same problem as builds . rssLatest is even worse: it will load every build of a job (not just newBuilds ) unless and until it finds one associated with this user.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/util/RunList.java
            http://jenkins-ci.org/commit/jenkins/a586609c4a2f2083a7f40a6055776afbf2b1f1c7
            Log:
            JENKINS-20892 Noting that RunList.getFirstBuild (used by BuildTimelineWidget) breaks lazy loading.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/util/RunList.java http://jenkins-ci.org/commit/jenkins/a586609c4a2f2083a7f40a6055776afbf2b1f1c7 Log: JENKINS-20892 Noting that RunList.getFirstBuild (used by BuildTimelineWidget) breaks lazy loading.
            Hide
            jglick Jesse Glick added a comment -

            For BuildTimelineWidget/control.jelly, it seems that DefaultEventSource.loadJSON can display new events incrementally, which would be compatible with ProgressiveRendering with some work. getFirstBuild must not be used, meaning that timeline_start cannot be set in advance, or can be set only to some very old value, or can be incrementally moved backward.

            Job/buildTimeTrend.jelly also uses BuildTimelineWidget, and anyway it iterates all builds of the job.

            Show
            jglick Jesse Glick added a comment - For BuildTimelineWidget/control.jelly , it seems that DefaultEventSource.loadJSON can display new events incrementally, which would be compatible with ProgressiveRendering with some work. getFirstBuild must not be used, meaning that timeline_start cannot be set in advance, or can be set only to some very old value, or can be incrementally moved backward. Job/buildTimeTrend.jelly also uses BuildTimelineWidget , and anyway it iterates all builds of the job.
            Hide
            jglick Jesse Glick added a comment -

            …and Job/buildTimeTrend.jelly calls Job.buildTimeGraph, which again iterates all builds (though asynchronously during an image load).

            Show
            jglick Jesse Glick added a comment - …and Job/buildTimeTrend.jelly calls Job.buildTimeGraph , which again iterates all builds (though asynchronously during an image load).
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #3095
            JENKINS-20892 Noting that RunList.getFirstBuild (used by BuildTimelineWidget) breaks lazy loading. (Revision a586609c4a2f2083a7f40a6055776afbf2b1f1c7)

            Result = SUCCESS
            Jesse Glick : a586609c4a2f2083a7f40a6055776afbf2b1f1c7
            Files :

            • core/src/main/java/hudson/util/RunList.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3095 JENKINS-20892 Noting that RunList.getFirstBuild (used by BuildTimelineWidget) breaks lazy loading. (Revision a586609c4a2f2083a7f40a6055776afbf2b1f1c7) Result = SUCCESS Jesse Glick : a586609c4a2f2083a7f40a6055776afbf2b1f1c7 Files : core/src/main/java/hudson/util/RunList.java
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/Functions.java
            core/src/main/java/hudson/model/BuildTimelineWidget.java
            core/src/main/java/hudson/model/Job.java
            core/src/main/java/hudson/model/User.java
            core/src/main/java/hudson/util/RunList.java
            core/src/main/java/jenkins/widgets/BuildListTable.java
            core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly
            core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly
            core/src/main/resources/hudson/model/User/builds.jelly
            core/src/main/resources/lib/hudson/buildListTable.jelly
            http://jenkins-ci.org/commit/jenkins/c5e23739fa85a6fe93ad90a35da4051a2767c874
            Log:
            JENKINS-20892 Partial fix of poor scalability in /builds and related displays.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/Functions.java core/src/main/java/hudson/model/BuildTimelineWidget.java core/src/main/java/hudson/model/Job.java core/src/main/java/hudson/model/User.java core/src/main/java/hudson/util/RunList.java core/src/main/java/jenkins/widgets/BuildListTable.java core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly core/src/main/resources/hudson/model/User/builds.jelly core/src/main/resources/lib/hudson/buildListTable.jelly http://jenkins-ci.org/commit/jenkins/c5e23739fa85a6fe93ad90a35da4051a2767c874 Log: JENKINS-20892 Partial fix of poor scalability in /builds and related displays.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/Functions.java
            core/src/main/java/hudson/model/BuildTimelineWidget.java
            core/src/main/java/hudson/model/Job.java
            core/src/main/java/hudson/model/User.java
            core/src/main/java/hudson/util/RunList.java
            core/src/main/java/jenkins/widgets/BuildListTable.java
            core/src/main/java/jenkins/widgets/BuildTimeTrend.java
            core/src/main/java/jenkins/widgets/RunListProgressiveRendering.java
            core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly
            core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly
            core/src/main/resources/hudson/model/User/builds.jelly
            core/src/main/resources/lib/hudson/buildListTable.jelly
            http://jenkins-ci.org/commit/jenkins/67b163321b6bd9b072b281348864867f90a0ba10
            Log:
            [FIXED JENKINS-20892] /

            {view,computer,user}

            //builds & /job//buildTimeTrend block HTTP response on build record loading.
            Merge branch 'lazy-build-lists-JENKINS-20892'

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/Functions.java core/src/main/java/hudson/model/BuildTimelineWidget.java core/src/main/java/hudson/model/Job.java core/src/main/java/hudson/model/User.java core/src/main/java/hudson/util/RunList.java core/src/main/java/jenkins/widgets/BuildListTable.java core/src/main/java/jenkins/widgets/BuildTimeTrend.java core/src/main/java/jenkins/widgets/RunListProgressiveRendering.java core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly core/src/main/resources/hudson/model/User/builds.jelly core/src/main/resources/lib/hudson/buildListTable.jelly http://jenkins-ci.org/commit/jenkins/67b163321b6bd9b072b281348864867f90a0ba10 Log: [FIXED JENKINS-20892] / {view,computer,user} / /builds & /job/ /buildTimeTrend block HTTP response on build record loading. Merge branch 'lazy-build-lists- JENKINS-20892 '
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            changelog.html
            core/src/main/java/hudson/util/RunList.java
            http://jenkins-ci.org/commit/jenkins/9679a6123ed30cb70e4d6e4b389f14d04746b995
            Log:
            JENKINS-20892 Noting merge of #1046.

            Compare: https://github.com/jenkinsci/jenkins/compare/0c2b72269925...9679a6123ed3

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/util/RunList.java http://jenkins-ci.org/commit/jenkins/9679a6123ed30cb70e4d6e4b389f14d04746b995 Log: JENKINS-20892 Noting merge of #1046. Compare: https://github.com/jenkinsci/jenkins/compare/0c2b72269925...9679a6123ed3
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #3106
            JENKINS-20892 Partial fix of poor scalability in /builds and related displays. (Revision c5e23739fa85a6fe93ad90a35da4051a2767c874)
            JENKINS-20892 Noting merge of #1046. (Revision 9679a6123ed30cb70e4d6e4b389f14d04746b995)

            Result = SUCCESS
            Jesse Glick : c5e23739fa85a6fe93ad90a35da4051a2767c874
            Files :

            • core/src/main/java/hudson/Functions.java
            • core/src/main/java/hudson/model/BuildTimelineWidget.java
            • core/src/main/resources/hudson/model/User/builds.jelly
            • core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly
            • core/src/main/java/hudson/model/Job.java
            • core/src/main/java/hudson/util/RunList.java
            • core/src/main/java/hudson/model/User.java
            • core/src/main/resources/lib/hudson/buildListTable.jelly
            • core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly
            • core/src/main/java/jenkins/widgets/BuildListTable.java

            Jesse Glick : 9679a6123ed30cb70e4d6e4b389f14d04746b995
            Files :

            • core/src/main/java/hudson/util/RunList.java
            • changelog.html
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3106 JENKINS-20892 Partial fix of poor scalability in /builds and related displays. (Revision c5e23739fa85a6fe93ad90a35da4051a2767c874) JENKINS-20892 Noting merge of #1046. (Revision 9679a6123ed30cb70e4d6e4b389f14d04746b995) Result = SUCCESS Jesse Glick : c5e23739fa85a6fe93ad90a35da4051a2767c874 Files : core/src/main/java/hudson/Functions.java core/src/main/java/hudson/model/BuildTimelineWidget.java core/src/main/resources/hudson/model/User/builds.jelly core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly core/src/main/java/hudson/model/Job.java core/src/main/java/hudson/util/RunList.java core/src/main/java/hudson/model/User.java core/src/main/resources/lib/hudson/buildListTable.jelly core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly core/src/main/java/jenkins/widgets/BuildListTable.java Jesse Glick : 9679a6123ed30cb70e4d6e4b389f14d04746b995 Files : core/src/main/java/hudson/util/RunList.java changelog.html
            Hide
            domi Dominik Bartholdi added a comment -

            is it possible that this is related to JENKINS-21159 ?

            Show
            domi Dominik Bartholdi added a comment - is it possible that this is related to JENKINS-21159 ?
            Hide
            jglick Jesse Glick added a comment -

            @domi if that is a regression. The issue does not say.

            Show
            jglick Jesse Glick added a comment - @domi if that is a regression. The issue does not say.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/util/RunList.java
            http://jenkins-ci.org/commit/jenkins/537e94a09c8fa88607d998162b5a62eac3fe11d5
            Log:
            JENKINS-20892 Noting that RunList.getFirstBuild (used by BuildTimelineWidget) breaks lazy loading.
            (cherry picked from commit a586609c4a2f2083a7f40a6055776afbf2b1f1c7)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/util/RunList.java http://jenkins-ci.org/commit/jenkins/537e94a09c8fa88607d998162b5a62eac3fe11d5 Log: JENKINS-20892 Noting that RunList.getFirstBuild (used by BuildTimelineWidget) breaks lazy loading. (cherry picked from commit a586609c4a2f2083a7f40a6055776afbf2b1f1c7)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/model/User.java
            http://jenkins-ci.org/commit/jenkins/46d4a638182d149b105414eadf172157e4fcb96e
            Log:
            JENKINS-20892 /user//rssLatest should use getAllItems to check in folders, just like /user//builds does.
            (cherry picked from commit ece93739dd291d03c673664afe6b4aaa00f07ef1)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/User.java http://jenkins-ci.org/commit/jenkins/46d4a638182d149b105414eadf172157e4fcb96e Log: JENKINS-20892 /user/ /rssLatest should use getAllItems to check in folders, just like /user/ /builds does. (cherry picked from commit ece93739dd291d03c673664afe6b4aaa00f07ef1)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/model/User.java
            http://jenkins-ci.org/commit/jenkins/915000b5a0a4c33e02104e47c4b9420192531e07
            Log:
            JENKINS-20892 Introduced method so that /user//rssLatest can check UserIdCause just like /user//builds already was.
            (cherry picked from commit 76af02eed23f43ebd7fde094d939339705b1ef0f)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/User.java http://jenkins-ci.org/commit/jenkins/915000b5a0a4c33e02104e47c4b9420192531e07 Log: JENKINS-20892 Introduced method so that /user/ /rssLatest can check UserIdCause just like /user/ /builds already was. (cherry picked from commit 76af02eed23f43ebd7fde094d939339705b1ef0f)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            core/src/main/java/hudson/Functions.java
            core/src/main/java/hudson/model/BuildTimelineWidget.java
            core/src/main/java/hudson/model/Job.java
            core/src/main/java/hudson/model/User.java
            core/src/main/java/hudson/util/RunList.java
            core/src/main/java/jenkins/widgets/BuildListTable.java
            core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly
            core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly
            core/src/main/resources/hudson/model/User/builds.jelly
            core/src/main/resources/lib/hudson/buildListTable.jelly
            http://jenkins-ci.org/commit/jenkins/ef6343be19fc990e0fcdab31d4d3f30c56baba0b
            Log:
            JENKINS-20892 Partial fix of poor scalability in /builds and related displays.
            (cherry picked from commit c5e23739fa85a6fe93ad90a35da4051a2767c874)

            Compare: https://github.com/jenkinsci/jenkins/compare/f61086b56cec...ef6343be19fc

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/Functions.java core/src/main/java/hudson/model/BuildTimelineWidget.java core/src/main/java/hudson/model/Job.java core/src/main/java/hudson/model/User.java core/src/main/java/hudson/util/RunList.java core/src/main/java/jenkins/widgets/BuildListTable.java core/src/main/resources/hudson/model/BuildTimelineWidget/control.jelly core/src/main/resources/hudson/model/Job/buildTimeTrend.jelly core/src/main/resources/hudson/model/User/builds.jelly core/src/main/resources/lib/hudson/buildListTable.jelly http://jenkins-ci.org/commit/jenkins/ef6343be19fc990e0fcdab31d4d3f30c56baba0b Log: JENKINS-20892 Partial fix of poor scalability in /builds and related displays. (cherry picked from commit c5e23739fa85a6fe93ad90a35da4051a2767c874) Compare: https://github.com/jenkinsci/jenkins/compare/f61086b56cec...ef6343be19fc
            Hide
            domi Dominik Bartholdi added a comment -

            with this fix/change, only 20 builds are shown in the buildTimeTrend - see JENKINS-22008 for more about it.

            Show
            domi Dominik Bartholdi added a comment - with this fix/change, only 20 builds are shown in the buildTimeTrend - see JENKINS-22008 for more about it.
            Hide
            samxiao Sam Xiao added a comment -

            Can we port this back to LTS?

            Show
            samxiao Sam Xiao added a comment - Can we port this back to LTS?

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: