Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63655

Performance degradation when running multiple stages in parallel


      Simple steps are taking too long to be finished when we are running multiple stages (10+) in parallel (assuming that each stage creates a pod).

      A step like a simple `which` command that under normal conditions should take less than 1s is taking 40s. I attached 2 pictures, one when we are running only 4 stages in parallel and the other when there are 30 stages. Note in the other image attached (conflicting_times) that according to the logs a task runs in ~6s however Jenkins takes 27s to recognize it as done.

      The same pipeline with 70 parallel stages running on static pods (without using the plugin) on Kubernetes with the same Docker image takes ~7 min to be completed. Whereas this pipeline with the same 70 stages, is taking ~25 minutes under elastic agents (Kubernetes Plugin).

      I've attached the JNLP container logs as well although it does not have much information.

        1. 4_parallel_stages.png
          44 kB
        2. 50_parallel_stages.png
          46 kB
        3. conflicting times.png
          conflicting times.png
          70 kB
        4. jnlp_logs.txt
          2 kB

            Unassigned Unassigned
            mateusvtt Mateus Tanure
            0 Vote for this issue
            5 Start watching this issue