Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42993

The activity page will hang if too many jobs are run

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • blueocean-plugin

      Problem

      When looking at the activity view when there are lots of jobs running the page will eventually lockup. 

      Steps to Reproduce

      This will setup a job that will run all the time, one instance of the job after the next.

      This is the pipeline script I was using but I am fairly sure that it doesn't matter all that much what the job is doing other than for timing:

      stage('one') {
       echo 'step one'
       }
       stage('two') {
       echo 'step two'
       }
       stage('three') {
       sleep 1
       echo 'step three'
       }
       stage('four') {
       echo 'step four'
       }
      

      Next go to the Blue Ocean activity view and wait for a few minutes. (This goes very fast if you remove the sleep.)

      After those few minutes your browser window should stop responding... And you may see that the browser process is using up a large amount of CPU time.

       

      Notes

      I can split this up into three tickets if you guys want. I am not sure how related they are at this point.

      When I slowed down the job run time with the above sleep the page still locked up which seems to imply a problem with the number of items not the rate that the items are updating. 

      When I slowed down the job I also noticed that the top line item's time kept increasing even though the reported time for the job when it finished was constant. 

      Finally, I noticed that there was a non-zero rate of jobs failing to update in BO. This may be related to the above issue or it could be that the number of jobs run surfaces the issue. Here is a screen cap of the issue in the UI:

          [JENKINS-42993] The activity page will hang if too many jobs are run

          James Dumay added a comment -

          I was able to reproduce this. Looking at the browser memory usage it seems that the data collected is never being purged when it reached a certain threshold.

          James Dumay added a comment - I was able to reproduce this. Looking at the browser memory usage it seems that the data collected is never being purged when it reached a certain threshold.

          James Dumay added a comment -

          cliffmeyers I suspect this is the browser running out of memory. Should we change BunkerService to store data in an LRU cache?

          James Dumay added a comment - cliffmeyers I suspect this is the browser running out of memory. Should we change BunkerService to store data in an LRU cache?

          Cliff Meyers added a comment -

          We need to profile it from both a CPU and RAM perspective. If CPU usage is spiking, it's more likely because of runaway renders or maybe the SVG animations are pummeling the C/GPU. We also have real-time updates of the Duration that each update independently.. that could be a little too heavy. I am a little skeptical that the in-memory object model is causing this, although it's possible those objects are huge if there are a lot of actions.

          Cliff Meyers added a comment - We need to profile it from both a CPU and RAM perspective. If CPU usage is spiking, it's more likely because of runaway renders or maybe the SVG animations are pummeling the C/GPU. We also have real-time updates of the Duration that each update independently.. that could be a little too heavy. I am a little skeptical that the in-memory object model is causing this, although it's possible those objects are huge if there are a lot of actions.

          James Dumay added a comment -

          We did drop all the actions from the REST unless you ask for them via the tree param so if this is a memory thing this may not be a huge issue anymore.

          James Dumay added a comment - We did drop all the actions from the REST unless you ask for them via the tree param so if this is a memory thing this may not be a huge issue anymore.

          Cliff Meyers added a comment -

          Just let me know when you think is the appropriate time to take on this bug and I'll be happy to look.

          Cliff Meyers added a comment - Just let me know when you think is the appropriate time to take on this bug and I'll be happy to look.

            Unassigned Unassigned
            zjoyner Zachary Joyner
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: