Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40934

LogActionImpl listener inefficient; poor performance queuing large parallel workloads

      When the number of parallel branches in pipeline increases a lot, the performance of the pipeline decreases significantly. There appears to be an n^2 algorithm somewhere.

      ~200 branches - okay
      4k branches - glacial

      Attached is a stack trace taken while the behavior was exibited. I'm pretty sure this ends up n^2:

      "Running CpsFlowExecutionOwner[corex_full/54:corex_full #54]" Id=421 Group=main RUNNABLE
      at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:703)

      • locked org.jenkinsci.plugins.workflow.cps.CpsFlowExecution@6ebdc0a3
        at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl.isRunning(LogActionImpl.java:152)
        at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl.access$000(LogActionImpl.java:66)
        at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl$1.onNewHead(LogActionImpl.java:93)
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1081)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.notifyNewHead(CpsThreadGroup.java:381)
        at org.jenkinsci.plugins.workflow.cps.FlowHead.setNewHead(FlowHead.java:119)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:171)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:126)
        at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)

      The code walks the heads every time a new head is added.

          [JENKINS-40934] LogActionImpl listener inefficient; poor performance queuing large parallel workloads

          Matthew Mitchell created issue -
          Jesse Glick made changes -
          Labels New: performance
          Jesse Glick made changes -
          Link New: This issue relates to JENKINS-38381 [ JENKINS-38381 ]
          Jesse Glick made changes -
          Component/s New: workflow-support-plugin [ 21719 ]
          Component/s Original: workflow-cps-plugin [ 21713 ]
          Summary Original: Poor performance queuing large parallel workloads New: LogActionImpl listener inefficient; poor performance queuing large parallel workloads

          Jesse Glick added a comment -

          Similar code was copied to JENKINS-38381.

          Jesse Glick added a comment - Similar code was copied to JENKINS-38381 .
          Jesse Glick made changes -
          Link New: This issue blocks JENKINS-38381 [ JENKINS-38381 ]
          Jesse Glick made changes -
          Link Original: This issue relates to JENKINS-38381 [ JENKINS-38381 ]
          Jesse Glick made changes -
          Link New: This issue is duplicated by JENKINS-42937 [ JENKINS-42937 ]
          Jesse Glick made changes -
          Assignee New: Jesse Glick [ jglick ]
          Jesse Glick made changes -
          Assignee Original: Jesse Glick [ jglick ] New: Sam Van Oort [ svanoort ]

            svanoort Sam Van Oort
            mmitche Matthew Mitchell
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: