Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58898

Stager mass-parallel builds

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • workflow-cps-plugin
    • None

      When running a mass-parallel build (~300 branches of pipeline in parallel) there is a noticeable slow down of the Jenkins UI.

       

      We have enough k8s nodes to support this, but master still gets slow in the beginning when the worker containers are starting up. This is due to each worker occupying a thread on master and hundreds of busy threads is not ideal.

       

      Would it be possible to do something like this:

      parallel parallelNodes, stager: '1s'

      This would tell the main job's thread to wait for 1s after each parallel job start before spawning a new one. This way the master would spin up threads on master slower and we can tune the stager parameter to not visibly slow Jenkins UI at the time of starting mass parallel job.

            Unassigned Unassigned
            voronaam Aleksey Vorona
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: