Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48868

blue ocean dashboard taking more than 50s to load.

    XMLWordPrintable

Details

    • Blue Ocean 1.5 - beta 1

    Description

      The dashboard is taking more than 50s to load and sometimes throws back 502 error. The error in log is below : - 

      // code placeholder
      Error while serving https://jenkins.XXXXXXXX.com/blue/rest/search/
      java.lang.NullPointerException
      	at io.jenkins.blueocean.rest.ApiHead.search(ApiHead.java:60)
      	at java.lang.invoke.MethodHandle.invokeWithArguments(Unknown Source)
      	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343)
      Caused: java.lang.reflect.InvocationTargetException
      	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:347)
      	at org.kohsuke.stapler.ForwardingFunction.invoke(ForwardingFunction.java:63)
      	at io.jenkins.blueocean.rest.pageable.PagedResponse$Processor.invoke(PagedResponse.java:42)
      	at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
      	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184)
      	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:117)
      	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:129)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
      	at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:686)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
      	at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
      	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
      	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:225)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
      	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
      	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
      	at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
      	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
      	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
      	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
      	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
      	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
      	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.Server.handle(Server.java:564)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
      	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
      	at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)
      	at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
      	at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.lang.Thread.run(Unknown Source)
      

      Attachments

        Issue Links

          Activity

            vivek Vivek Pandey added a comment -

            jhoblitt Right, need to check how GitHub commits appear on JIRA. michaelneale do you know how to get commits appear on JIRA tickets? I think if we at least update JIRA with PR being worked on is not bad either.

            vivek Vivek Pandey added a comment - jhoblitt  Right, need to check how GitHub commits appear on JIRA. michaelneale do you know how to get commits appear on JIRA tickets? I think if we at least update JIRA with PR being worked on is not bad either.

            That sounds very plausible and being related to favorites would explain why my production env seems slower than test instances.

            No code was attached to the ticket – is jira supposed to be doing that for the blueocean repo?

            jhoblitt Joshua Hoblitt added a comment - That sounds very plausible and being related to favorites would explain why my production env seems slower than test instances. No code was attached to the ticket – is jira supposed to be doing that for the blueocean repo?
            vivek Vivek Pandey added a comment -

            jhoblitt, sorry not enough info in this ticket.

             

            Tl;dr version: 

            Fix was to return run summary in latestRun instead of full Run object graph. See https://github.com/jenkinsci/blueocean-plugin/pull/1635.

             

            More details:

            Dashbaord lists pipelines and favorites. In both cases each pipeline also includes 'latestRun' property that includes complete Run object. Run objects can be very expensive and includes expensive objects such as changeSets and many others. Further, it was validated to be the case going over HAR files submitted by users on this ticket and others. 

            Favorites can be expensive too, specially it was resulting in to expensive computation and fact that we autofavorite and need all favorites so that we can show in pipeline/branch/activities pages - had more involved fixes. This is fixed as well, see PR https://github.com/jenkinsci/blueocean-plugin/pull/1638.

             

            Both PRs are merged to master. If you can, please give it a try. By the end of this week or early next week we are going to release 1.5-beta-1 and that should have these fixes as well.

            vivek Vivek Pandey added a comment - jhoblitt , sorry not enough info in this ticket.   Tl;dr version:  Fix was to return run summary in latestRun instead of full Run object graph. See https://github.com/jenkinsci/blueocean-plugin/pull/1635.   More details: Dashbaord lists pipelines and favorites. In both cases each pipeline also includes 'latestRun' property that includes complete Run object. Run objects can be very expensive and includes expensive objects such as changeSets and many others. Further, it was validated to be the case going over HAR files submitted by users on this ticket and others.  Favorites can be expensive too, specially it was resulting in to expensive computation and fact that we autofavorite and need all favorites so that we can show in pipeline/branch/activities pages - had more involved fixes. This is fixed as well, see PR https://github.com/jenkinsci/blueocean-plugin/pull/1638.   Both PRs are merged to master. If you can, please give it a try. By the end of this week or early next week we are going to release 1.5-beta-1 and that should have these fixes as well.

            What was the resolution?

            jhoblitt Joshua Hoblitt added a comment - What was the resolution?

            I've attached a cut'n'paste of the output from the /threadDump end point – is there a better way to do this?

            jhoblitt Joshua Hoblitt added a comment - I've attached a cut'n'paste of the output from the /threadDump end point – is there a better way to do this?

            People

              vivek Vivek Pandey
              xs2bharat Bharat Bhatia
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: