Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3922

Slave is slow copying maven artifacts to master

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Critical Critical
    • remoting
    • None
    • Platform: All, OS: All

      The artifact transfer is currently a 3-4x penalty for the project that I am
      working on. I have reproduced the issue with a simple test pom that does
      nothing but jar hudson.war. I performed this test on a heterogeneous
      environment. Both master and slave are running Fedora 10, but the master is a
      faster machine. Still, it highlights the issue.

      Here are some stats (all stats are after caching dependencies in the local repos):
      Master build through Hudson: 19s
      Master build from command line (no Hudson): 9s
      Slave build through Hudson: 1m46s
      Slave build from command line (no Hudson): 16s

      To be fair we should at least add time to do a straight scp of the artifact from
      slave to master. The two nodes share a 100 Mbit switch:

      $ scp target/slow-rider-1.0.0-SNAPSHOT.jar master_node:
      slow-rider-1.0.0NAPSHOT.jar 100% 25MB 12.7MB/s 00:02

      Of course this example exaggerates the issue to make it more clear but not by
      too much. I originally noticed this in a completely separate environment that
      was all virtual. I reproduced this on two physical machines using a different
      switch and different ethernet drivers (both virtual and physical). The
      reproducibility plus the comparison against command line + scp leads me to
      suspect eager flushing.

            kohsuke Kohsuke Kawaguchi
            pamdirac John McNair
            Votes:
            35 Vote for this issue
            Watchers:
            34 Start watching this issue

              Created:
              Updated:
              Resolved: