Loading...

XML

Word

Printable

Type: Bug
Resolution: Fixed
Priority: Critical
Component/s: core, workflow-durable-task-step-plugin, workflow-support-plugin
Labels:
- performance
- pipeline
Environment:

Hide
Jenkins 2.69
Pipeline 2.5

Ubuntu 16.04
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)
JVM args: -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:+UnlockDiagnosticVMOptions -XX:G1SummarizeRSetStatsPeriod=1 -server -XX:+AlwaysPreTouch -Djenkins.install.runSetupWizard=false -Dgroovy.use.classvalue=true -Xmx8192m -Xms8192m

Show
Jenkins 2.69 Pipeline 2.5 Ubuntu 16.04 OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode) JVM args: -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:+UnlockDiagnosticVMOptions -XX:G1SummarizeRSetStatsPeriod=1 -server -XX:+AlwaysPreTouch -Djenkins.install.runSetupWizard=false -Dgroovy.use.classvalue=true -Xmx8192m -Xms8192m

Similar Issues:

Show

Execution of parallel blocks scales poorly for values of N > 100. With ~50 nodes (each with 4 executors, for a total of ~200 slots), the following pipeline job takes extraordinarily long to execute:

def stepsForParallel = [:]
for (int i = 0; i < Integer.valueOf(params.SUB_JOBS); i++) {
  def s = "subjob_${i}" 
  stepsForParallel[s] = {
    node("darwin") {
      echo "hello"
    }
  }
}
parallel stepsForParallel

SUB_JOBS   Time (sec)
---------------------
 100         10
 200         40
 300         96
 400        214
 500        392
 600        660
 700        960
 800       1500
 900       2220
1000       gave up...

At no point does the underlying system become taxed (CPU utilization is very low, as this is a very beefy system – 28 cores, 128GB RAM, SSDs)

CPU and Thread CPU Time Sampling (via VisualVM) are attached for reference.