Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18377

Improve the performance when listing many jobs (GSoC 2019, coding phase 2)

    XMLWordPrintable

    Details

    • Epic Name:
      RoleStrategy-Performance
    • Similar Issues:

      Description

      This affects a jenkins installation with 750+ jobs.

      When loading the "Overview" to list all jobs, the loading time is over 60 seconds.

      With the role-strategy plugin disabled, the loading time goes down to 5 seconds.

        Attachments

          Issue Links

          duplicates

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. JENKINS-58203 [GSoC] Role Strategy Plugin Performance Improvements: Phase 2

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-18723 Relative links to job are built very slowly

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-18721 ListView asks for all jobs too often

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-20475 Add option to skip security checks for System users

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-17122 SidACL._hasPermission does too much object construction

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-25075 Weather column should render asynchronously

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-55045 performance optimization

          • Major - Major loss of function.
          • Open
          links to

          Web Link CloudBees Internal OSS-103

            Activity

            Ascending order - Click to sort in descending order
            25 older comments
            Hide
            Permalink
            oleg_nenashev Oleg Nenashev added a comment -

            Abhyudaya Sharma is currently working on this EPIC as a part of hist GSoC project. https://jenkins.io/projects/gsoc/2019/role-strategy-performance/ .Some stories have been already addressed in the master branch.

            Abhyudaya Sharma could you please take a look at the EPIc and update tasks or add missing ones?

            Show
            oleg_nenashev Oleg Nenashev added a comment - Abhyudaya Sharma is currently working on this EPIC as a part of hist GSoC project.  https://jenkins.io/projects/gsoc/2019/role-strategy-performance/  .Some stories have been already addressed in the master branch. Abhyudaya Sharma could you please take a look at the EPIc and update tasks or add missing ones?
            Hide
            Permalink
            runzexia runze xia added a comment -

            I thought about our authentication model. First we need to get the acl (traversing all the roles), and in the second step to determine whether the role in the acl has the appropriate permissions.
            Can we simplify the acquisition of acl content, we construct the roleMap in the second step, so we can reduce the traversal of some roles.

            This idea may require redesigning the ACL object. The way the ACL is cached may also change.

            Show
            runzexia runze xia added a comment - I thought about our authentication model. First we need to get the acl (traversing all the roles), and in the second step to determine whether the role in the acl has the appropriate permissions. Can we simplify the acquisition of acl content, we construct the roleMap in the second step, so we can reduce the traversal of some roles. This idea may require redesigning the ACL object. The way the ACL is cached may also change.
            Hide
            Permalink
            abhyudaya Abhyudaya Sharma added a comment -

            runze xia I have just committed https://github.com/jenkinsci/role-strategy-plugin/pull/89/commits/a02ea97b0bd512da1973f502c6ab3faca69845d8 for the folder based authorization. This changes how global roles work. Is it something like what you're looking for?

            Show
            abhyudaya Abhyudaya Sharma added a comment - runze xia  I have just committed  https://github.com/jenkinsci/role-strategy-plugin/pull/89/commits/a02ea97b0bd512da1973f502c6ab3faca69845d8  for the folder based authorization. This changes how global roles work. Is it something like what you're looking for?
            Hide
            Permalink
            runzexia runze xia added a comment -

            Abhyudaya Sharma   Yes, it is very similar to the approach I mentioned. If we operate in havePermission func, can we reduce some memory consumption? And we might be very simple to break the loop?

            Show
            runzexia runze xia added a comment - Abhyudaya Sharma    Yes, it is very similar to the approach I mentioned. If we operate in havePermission func, can we reduce some memory consumption? And we might be very simple to break the loop?
            Hide
            Permalink
            jglick Jesse Glick added a comment -

            When loading the "Overview" to list all jobs, the loading time is over 60 seconds.

            Certainly it is good to optimize particular bottlenecks as we find them, but in tandem I would suggest changing the rendering of the dashboard to be progressive so that server-side bottlenecks do not completely prevent display nor block the user from initiating unrelated actions (like clicking on the sidebar). JENKINS-25075 suggests doing this for individual *ListView/column.jelly cells, though you could just as easily render whole rows lazily, ameliorating a bunch of performance problems at once.

            Show
            jglick Jesse Glick added a comment - When loading the "Overview" to list all jobs, the loading time is over 60 seconds. Certainly it is good to optimize particular bottlenecks as we find them, but in tandem I would suggest changing the rendering of the dashboard to be progressive so that server-side bottlenecks do not completely prevent display nor block the user from initiating unrelated actions (like clicking on the sidebar). JENKINS-25075 suggests doing this for individual *ListView/column.jelly cells, though you could just as easily render whole rows lazily, ameliorating a bunch of performance problems at once.

              People

              Assignee:
              abhyudaya Abhyudaya Sharma
              Reporter:
              croesus Croesus Kall
              Votes:
              14 Vote for this issue
              Watchers:
              26 Start watching this issue

                Dates

                Created:
                Updated: