Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-23293

Load balancing does not work when many jobs are launched simultaneously

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • leastload-plugin
    • None

      We have a master job (using the Multijob plugin) which launches eight sub-jobs in our CI pipeline. We have installed the least load plugin to evenly distribute the load on our (currently two) Jenkins slaves. However, we often see that seven jobs are launched on one slave and only one on the other.

      My guess is that since the jobs are launched at exactly the same time, the least load plugin doesn't have time to see the new jobs launched on the slaves when it gets requests to map the other jobs, thus placing them all on the same slave.

      If there's no cleaner solution available, then I'd suggest that the plugin would remember the jobs it has mapped to a particular slave for a few seconds, and factor them into the load balancing decisions within that time frame.

          [JENKINS-23293] Load balancing does not work when many jobs are launched simultaneously

          James Ladan added a comment -

          We're using a Gerrit monitor, and when someone pushes in more than one patch at a time, the jobs are not distributed evenly.

          James Ladan added a comment - We're using a Gerrit monitor, and when someone pushes in more than one patch at a time, the jobs are not distributed evenly.

          Jifeng Zhang added a comment -

          I have tried to make a possible fix for this issue. When jobs launched from Multijob plugin or many sub jobs triggered at the same time by a "master" job, computer.countIdle() does not return the correct idling number of executors. I tried to replace it with ExecutorChunk's size, which I suppose it always represent the idling executors on a computer/slave. But I am not sure this is the correct solution. Any comment on the pull request is appreciated.

          https://github.com/jenkinsci/leastload-plugin/pull/2

          Jifeng Zhang added a comment - I have tried to make a possible fix for this issue. When jobs launched from Multijob plugin or many sub jobs triggered at the same time by a "master" job, computer.countIdle() does not return the correct idling number of executors. I tried to replace it with ExecutorChunk's size, which I suppose it always represent the idling executors on a computer/slave. But I am not sure this is the correct solution. Any comment on the pull request is appreciated. https://github.com/jenkinsci/leastload-plugin/pull/2

            Unassigned Unassigned
            plaa Sampo Niskanen
            Votes:
            4 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: