Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70808

Severe contention on io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeImpl.isRestartable with large Declarative pipelines

XMLWordPrintable

    • blue-ocean:1.27.3

      (this happens in the context of declarative pipelines, that have a restartable action)

      While troubleshooting environment with large declarative pipeline, Blue Ocean causes some severe contention issues when users are looking at pages for which the stacktrace look like :

      Handling GET /blue/rest/organizations/jenkins/pipelines/mypipeline/runs/123456/nodes/ from 127.0.0.1 : Jetty (winstone)-787028
      java.lang.Thread.State: BLOCKED (on object monitor)
          at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1027)
          - waiting to lock <0x0000000256ad8ae0> (a java.util.concurrent.ConcurrentHashMap$Node)
          at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$Timing.close(CpsFlowExecution.java:424)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$TimingFlowNodeStorage.getNode(CpsFlowExecution.java:1851)
          at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getNode(CpsFlowExecution.java:1178)
          at org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView.findEnclosingBlockStart(StandardGraphLookupView.java:187)
          at org.jenkinsci.plugins.workflow.graph.GraphLookupView$EnclosingBlocksIterable$EnclosingBlocksIterator.next(GraphLookupView.java:80)
          at org.jenkinsci.plugins.workflow.graph.GraphLookupView$EnclosingBlocksIterable$EnclosingBlocksIterator.next(GraphLookupView.java:63)
          at org.jenkinsci.plugins.workflow.graphanalysis.FilteratorImpl.next(FilteratorImpl.java:75)
          at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.leastCommonAncestor(ForkScanner.java:403)
          at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.setHeads(ForkScanner.java:452)
          at org.jenkinsci.plugins.workflow.graphanalysis.AbstractFlowScanner.setup(AbstractFlowScanner.java:140)
          at org.jenkinsci.plugins.workflow.graphanalysis.AbstractFlowScanner.findFirstMatch(AbstractFlowScanner.java:250)
          at sun.reflect.GeneratedMethodAccessor2329.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
          at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
          at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:144)
          at org.jenkinsci.plugins.pipeline.modeldefinition.Utils.findStageFlowNodes(Utils.groovy:238)
          at org.jenkinsci.plugins.pipeline.modeldefinition.Utils$findStageFlowNodes$15.callStatic(Unknown Source)
          at org.jenkinsci.plugins.pipeline.modeldefinition.Utils.stageHasStatusOf(Utils.groovy:365)
          at org.jenkinsci.plugins.pipeline.modeldefinition.actions.RestartDeclarativePipelineAction.getRestartableStages(RestartDeclarativePipelineAction.java:162)
          at io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeImpl.isRestartable(PipelineNodeImpl.java:189)
          at java.lang.invoke.LambdaForm$DMH/1800959603.invokeVirtual_L_I(LambdaForm$DMH)
          at java.lang.invoke.LambdaForm$BMH/847639156.reinvoke(LambdaForm$BMH)
          at java.lang.invoke.LambdaForm$MH/985480433.invoke_MT(LambdaForm$MH)
          at io.jenkins.blueocean.commons.stapler.export.MethodProperty.getValue(MethodProperty.java:72)
          at io.jenkins.blueocean.commons.stapler.export.ExportInterceptor$1.getValue(ExportInterceptor.java:45)
          at io.jenkins.blueocean.commons.stapler.Export$BlueOceanExportInterceptor.getValue(Export.java:195)
          at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:136)
          at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:220)
          at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:216)
          at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:191)
          at io.jenkins.blueocean.commons.stapler.Export.writeOne(Export.java:176)
          at io.jenkins.blueocean.commons.stapler.Export.serveExposedBean(Export.java:164)
          at io.jenkins.blueocean.commons.stapler.Export.doJson(Export.java:107)
          at io.jenkins.blueocean.rest.pageable.PagedResponse$Processor.lambda$invoke$0(PagedResponse.java:80)
          at io.jenkins.blueocean.rest.pageable.PagedResponse$Processor$$Lambda$2558/141249776.generateResponse(Unknown Source)
      [...]
      

      Blue Ocean check for each flow node if it is restartable by checking on all stages:

      Which seems very expensive but above all hammers the getNode of the same CpsFlowExecution.

            allan_burdajewicz Allan BURDAJEWICZ
            allan_burdajewicz Allan BURDAJEWICZ
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: