Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40934

LogActionImpl listener inefficient; poor performance queuing large parallel workloads

    XMLWordPrintable

Details

    Description

      When the number of parallel branches in pipeline increases a lot, the performance of the pipeline decreases significantly. There appears to be an n^2 algorithm somewhere.

      ~200 branches - okay
      4k branches - glacial

      Attached is a stack trace taken while the behavior was exibited. I'm pretty sure this ends up n^2:

      "Running CpsFlowExecutionOwner[corex_full/54:corex_full #54]" Id=421 Group=main RUNNABLE
      at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:703)

      • locked org.jenkinsci.plugins.workflow.cps.CpsFlowExecution@6ebdc0a3
        at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl.isRunning(LogActionImpl.java:152)
        at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl.access$000(LogActionImpl.java:66)
        at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl$1.onNewHead(LogActionImpl.java:93)
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1081)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.notifyNewHead(CpsThreadGroup.java:381)
        at org.jenkinsci.plugins.workflow.cps.FlowHead.setNewHead(FlowHead.java:119)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:171)
        at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:126)
        at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)

      The code walks the heads every time a new head is added.

      Attachments

        Issue Links

          Activity

            mmitche Matthew Mitchell created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Labels performance
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-38381 [ JENKINS-38381 ]
            jglick Jesse Glick made changes -
            Component/s workflow-support-plugin [ 21719 ]
            Component/s workflow-cps-plugin [ 21713 ]
            Summary Poor performance queuing large parallel workloads LogActionImpl listener inefficient; poor performance queuing large parallel workloads
            jglick Jesse Glick made changes -
            Link This issue blocks JENKINS-38381 [ JENKINS-38381 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-38381 [ JENKINS-38381 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-42937 [ JENKINS-42937 ]
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ]
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ] Sam Van Oort [ svanoort ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 34 (Web Link)" [ 15739 ]
            jglick Jesse Glick made changes -
            Link This issue depends on JENKINS-38223 [ JENKINS-38223 ]
            svanoort Sam Van Oort made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-45553 [ JENKINS-45553 ]

            People

              svanoort Sam Van Oort
              mmitche Matthew Mitchell
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: