Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45146

High CPU load with BlueOcean Plugin

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: blueocean-plugin
    • Environment:
    • Similar Issues:
    • Epic Link:
    • Sprint:
      Blue Ocean 1.2-beta2, Blue Ocean 1.2-beta3, Blue Ocean 1.2-beta4, Blue Ocean 1.2, Blue Ocean 1.3, Blue Ocean 1.4 - beta 1

      Description

      At some point (presumably after triggering some jobs) our jenkins master node gets very high cpu load. After that the UI becomes not usable at all and we have to manually restart/kill the process.

       

      WARNUNG: Error publishing Run queued event. org.jenkinsci.plugins.pubsub.MessageException: Event enrichment failure due to unexpected exception in io.jenkins.blueocean.events.BlueMessageEnricher. at org.jenkinsci.plugins.pubsub.PubsubBus.publish(PubsubBus.java:135) at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.publish(SyncQueueListener.java:178) at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.publish(SyncQueueListener.java:172) at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.onEnterWaiting(SyncQueueListener.java:139) at hudson.model.Queue$WaitingItem.enter(Queue.java:2392) at hudson.model.Queue.scheduleInternal(Queue.java:632) at hudson.model.Queue.schedule2(Queue.java:588) at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:155) at jenkins.model.ParameterizedJobMixIn.scheduleBuild(ParameterizedJobMixIn.java:115) at jenkins.model.ParameterizedJobMixIn.scheduleBuild(ParameterizedJobMixIn.java:104) at com.cloudbees.jenkins.GitHubPushTrigger$1.run(GitHubPushTrigger.java:143) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at io.jenkins.blueocean.service.embedded.rest.QueueUtil.getQueuedItem(QueueUtil.java:24) at io.jenkins.blueocean.events.BlueMessageEnricher.enrich(BlueMessageEnricher.java:96) at org.jenkinsci.plugins.pubsub.PubsubBus.publish(PubsubBus.java:133) ... 16 more

       

      This is the only suspicious error/exception we found in our logs. The time the exception is thrown matches with the time the cpu load on the master node increases.

        Attachments

          Activity

          Hide
          jbochenski Jakub Bochenski added a comment -

          It seems we are affected by the same issue (high load, delays when adding jobs to queue, aforementioned stack trace).

          We don't have build-priority-plugin, only Blueocean

          Show
          jbochenski Jakub Bochenski added a comment - It seems we are affected by the same issue (high load, delays when adding jobs to queue, aforementioned stack trace). We don't have build-priority-plugin, only Blueocean
          Hide
          michaelneale Michael Neale added a comment -

          Ben Hines good to hear - did you see the issue on the secondary? 

           

          Jakub Bochenski ok - so it isn't priority plugin that is very good to know. As this is still seeming hard to reproduce, can you attach a support core bundle? https://wiki.jenkins.io/display/JENKINS/Support+Core+Plugin - that will help show what threads are doing things or not doing things etc. 

          Show
          michaelneale Michael Neale added a comment - Ben Hines good to hear - did you see the issue on the secondary?    Jakub Bochenski ok - so it isn't priority plugin that is very good to know. As this is still seeming hard to reproduce, can you attach a support core bundle? https://wiki.jenkins.io/display/JENKINS/Support+Core+Plugin  - that will help show what threads are doing things or not doing things etc. 
          Hide
          jbochenski Jakub Bochenski added a comment -

          Michael Neale will do next time this happens

          Show
          jbochenski Jakub Bochenski added a comment - Michael Neale will do next time this happens
          Hide
          michaelneale Michael Neale added a comment -

          Hrm it is possible this was similar to or same cause of https://issues.jenkins-ci.org/browse/JENKINS-47146

          (which has a fix pending rollout) 

          Show
          michaelneale Michael Neale added a comment - Hrm it is possible this was similar to or same cause of https://issues.jenkins-ci.org/browse/JENKINS-47146 (which has a fix pending rollout) 
          Hide
          michaelneale Michael Neale added a comment -

          I believe this is fixed via: https://github.com/jenkinsci/blueocean-plugin/pull/1433

           

          which is in 1.3 betas and soon to be GA

          Show
          michaelneale Michael Neale added a comment - I believe this is fixed via: https://github.com/jenkinsci/blueocean-plugin/pull/1433   which is in 1.3 betas and soon to be GA

            People

            Assignee:
            michaelneale Michael Neale
            Reporter:
            paulschmidt Paul Schmidt
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: