Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27650

Page loads slow with hundreds of throttled builds in queue

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      When there are hundreds of throttled builds in the queue, page loads increase by an order of magnitude.

      Steps to reproduce:

      1. Run Jenkins 1.580.2 and latest throttle-concurrent-builds plugin
      2. Create a matrix job with 200 combinations (attached)
      3. In the same job, select "Throttle Concurrent Builds" with a maximum of 7 builds throttled as part of a category called 'semaphore'
      4. Set number of executors on the 'master' queue to 200
      5. Run the job. There should only be 7 builds running due to the throttling

      Page load times will increase by an order of magnitude – I observed 10 seconds from 

      time curl http://localhost:8080/jenkins/ajaxBuildQueue

      If you remove the throttling in the job configuration, the page load times will be under 50 ms.

        Attachments

          Issue Links

          is blocking

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-19623 Jobs throttling causes performance issues in case of enabled security

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-31982 Jenkins UI hangs

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-27708 Concurrent build limits not honored on Jenkins 1.607

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-20046 Massive Jenkins slowdown when jobs in Queue (due to Queue.maintain())

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Docker demo to reproduce

          Web Link PR 27

            Activity

            Ascending order - Click to sort in descending order
            recampbell Ryan Campbell created issue -
            oleg_nenashev Oleg Nenashev made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-20046 [ JENKINS-20046 ]
            recampbell Ryan Campbell made changes -
            Attachment threaddump.txt [ 28908 ]
            jglick Jesse Glick made changes -
            Assignee Oleg Nenashev [ oleg_nenashev ] Jesse Glick [ jglick ]
            jglick Jesse Glick made changes -
            Description When there are hundreds of throttled builds in the queue, page loads increase by an order of magnitude.

            Steps to reproduce:

            # Run Jenkins 1.580.2 and latest throttle-concurrent-builds plugin
            # Create a matrix job with 200 combinations (attached)
            # In the same job, select "Throttle Concurrent Builds" with a maximum of 7 builds throttled as part of a category called 'semaphore'
            # Set number of executors on the 'master' queue to 200
            # Run the job. There should only be 7 builds running due to the throttling

            Page load times will increase by an order of magnitude -- I observed 10 seconds from {{{time curl http://localhost:8080/jenkins/ajaxBuildQueue}}}

            If you remove the throttling in the job configuration, the page load times will be under 50 ms.             		When there are hundreds of throttled builds in the queue, page loads increase by an order of magnitude.

            Steps to reproduce:

            # Run Jenkins 1.580.2 and latest throttle-concurrent-builds plugin
            # Create a matrix job with 200 combinations (attached)
            # In the same job, select "Throttle Concurrent Builds" with a maximum of 7 builds throttled as part of a category called 'semaphore'
            # Set number of executors on the 'master' queue to 200
            # Run the job. There should only be 7 builds running due to the throttling

            Page load times will increase by an order of magnitude -- I observed 10 seconds from {code:none}time curl http://localhost:8080/jenkins/ajaxBuildQueue{code}

            If you remove the throttling in the job configuration, the page load times will be under 50 ms.
            recampbell Ryan Campbell made changes -
            Attachment config.xml [ 29000 ]
            jglick Jesse Glick made changes -
            Labels performance performance queue
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "Docker demo to reproduce (Web Link)" [ 12302 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 27 (Web Link)" [ 12400 ]
            jglick Jesse Glick made changes -
            Link This issue is blocking JENKINS-19623 [ JENKINS-19623 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-27708 [ JENKINS-27708 ]
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ] Oleg Nenashev [ oleg_nenashev ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-31982 [ JENKINS-31982 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 161862 ] JNJira + In-Review [ 185585 ]
            oleg_nenashev Oleg Nenashev made changes -
            Status In Progress [ 3 ] Open [ 1 ]
            oleg_nenashev Oleg Nenashev made changes -
            Assignee Oleg Nenashev [ oleg_nenashev ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              recampbell Ryan Campbell
              Votes:
              12 Vote for this issue
              Watchers:
              20 Start watching this issue

                Dates

                Created:
                Updated: