Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19623

Jobs throttling causes performance issues in case of enabled security

XMLWordPrintable

      • QueueTaskDispatcher locks the queue
      • ThrottleQueueTaskDispatcher invokes Jenkins.Instance().getAllItems() method, which has a security check for the each item
      • Every dispatching attempt causes security check for all jobs in the system
      • Due to big delays, so that dispatching almost blocks the queue and web interfaces

      Workaround - use Hudson.getInstance().getItemMap(), which has not security checks

      "Jenkins cron thread" prio=10 tid=0x0000000005d28000 nid=0x667b runnable [0x0000000041df4000]
      java.lang.Thread.State: RUNNABLE
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleMap$2.perform(RoleMap.java:270)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleMap$RoleWalker.walk(RoleMap.java:327)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleMap$RoleWalker.<init>(RoleMap.java:316)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleMap$2.<init>(RoleMap.java:268)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleMap.getMatchingRoles(RoleMap.java:268)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleMap.newMatchingRoleMap(RoleMap.java:224)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy.getACL(RoleBasedAuthorizationStrategy.java:111)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy.getACL(RoleBasedAuthorizationStrategy.java:128)
      at com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy.getACL(RoleBasedAuthorizationStrategy.java:123)
      at hudson.model.Job.getACL(Job.java:1324)
      at hudson.model.AbstractItem.hasPermission(AbstractItem.java:448)
      at jenkins.model.Jenkins.getAllItems(Jenkins.java:1397)
      at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.getCategoryProjects(ThrottleQueueTaskDispatcher.java:211)
      at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:121)
      at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canTake(ThrottleQueueTaskDispatcher.java:36)
      at hudson.model.queue.QueueTaskDispatcher.canTake(QueueTaskDispatcher.java:101)
      at hudson.model.Queue$JobOffer.canTake(Queue.java:254)
      at hudson.model.Queue.maintain(Queue.java:1032)

      • locked <0x00000006805417d8> (a hudson.model.Queue)
        at hudson.model.Queue$MaintainTask.doRun(Queue.java:1762)
        at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
        at java.util.TimerThread.mainLoop(Timer.java:555)
        at java.util.TimerThread.run(Timer.java:505)

          [JENKINS-19623] Jobs throttling causes performance issues in case of enabled security

          Oleg Nenashev created issue -

          Oleg Nenashev added a comment -

          Upd. Issue affects categories only

          Oleg Nenashev added a comment - Upd. Issue affects categories only

          Marco Miller added a comment -

          Hi Oleg; we're interested in fixing this issue too (Ericsson).
          How is it going from your end, as we speak?
          Thx+ for letting me know! =)

          Marco Miller added a comment - Hi Oleg; we're interested in fixing this issue too (Ericsson). How is it going from your end, as we speak? Thx+ for letting me know! =)

          Oleg Nenashev added a comment -

          Hello Marco,

          Pull request is waiting for review (it has been created about month ago).
          https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/6
          Seems I need to send an another message to Andrew.

          The fix successfully runs on my installations, so we just need to merge it.

          Best regards,
          Oleg Nenashev

          Oleg Nenashev added a comment - Hello Marco, Pull request is waiting for review (it has been created about month ago). https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/6 Seems I need to send an another message to Andrew. The fix successfully runs on my installations, so we just need to merge it. Best regards, Oleg Nenashev
          Oleg Nenashev made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          Jesse Glick made changes -
          Link New: This issue is duplicated by JENKINS-16751 [ JENKINS-16751 ]
          Jesse Glick made changes -
          Assignee Original: Oleg Nenashev [ oleg_nenashev ] New: Jesse Glick [ jglick ]

          Jesse Glick added a comment -

          Working on a different fix.

          Jesse Glick added a comment - Working on a different fix.

          Jesse Glick added a comment -

          https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/10 offered as an alternative. (I have no installation to check it on, so relying on automated tests.)

          Jesse Glick added a comment - https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/10 offered as an alternative. (I have no installation to check it on, so relying on automated tests.)
          Jesse Glick made changes -
          Link New: This issue depends on JENKINS-20474 [ JENKINS-20474 ]

            jglick Jesse Glick
            oleg_nenashev Oleg Nenashev
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: