-
Bug
-
Resolution: Duplicate
-
Critical
-
None
-
Platform: All, OS: All
The artifact transfer is currently a 3-4x penalty for the project that I am
working on. I have reproduced the issue with a simple test pom that does
nothing but jar hudson.war. I performed this test on a heterogeneous
environment. Both master and slave are running Fedora 10, but the master is a
faster machine. Still, it highlights the issue.
Here are some stats (all stats are after caching dependencies in the local repos):
Master build through Hudson: 19s
Master build from command line (no Hudson): 9s
Slave build through Hudson: 1m46s
Slave build from command line (no Hudson): 16s
To be fair we should at least add time to do a straight scp of the artifact from
slave to master. The two nodes share a 100 Mbit switch:
$ scp target/slow-rider-1.0.0-SNAPSHOT.jar master_node:
slow-rider-1.0.0NAPSHOT.jar 100% 25MB 12.7MB/s 00:02
Of course this example exaggerates the issue to make it more clear but not by
too much. I originally noticed this in a completely separate environment that
was all virtual. I reproduced this on two physical machines using a different
switch and different ethernet drivers (both virtual and physical). The
reproducibility plus the comparison against command line + scp leads me to
suspect eager flushing.
- is related to
-
-
- Resolved
-
-
-
- Resolved
-
-
-
- Resolved
-
-
-
- Closed
-
[JENKINS-3922] Slave is slow copying maven artifacts to master
John McNair
created issue -
John McNair
added a comment - Oops. I just noticed that I forgot to upgrade this environment. The above
stats were collected on 309. On 312 we have:
Master: 20s
Slave: 1m18s
There seems to be definite improvement but still a big penalty for the slave.
John McNair
added a comment - Oops. I just noticed that I forgot to upgrade this environment. The above
stats were collected on 309. On 312 we have:
Master: 20s
Slave: 1m18s
There seems to be definite improvement but still a big penalty for the slave. Would it be possible for you to run a packet capturing tool like Wireshark to
obtain the network packet dump between the master and the slave?
| Status | Original: Open [ 1 ] | New: In Progress [ 3 ] |
John McNair
added a comment - You want the ssh traffic? Is that helpful? Also, it is ~63MB for this build.
Is there a subset that you would want to see?
John McNair
added a comment - You want the ssh traffic? Is that helpful? Also, it is ~63MB for this build.
Is there a subset that you would want to see? 
trodriguez
added a comment - watching this issue Just noting that this is still very definitely the case - and for what it's worth, I had the same speed problems both with the current MavenArtifact contents and with a test I did using FilePath.copyRecursiveTo instead of FilePath.copyTo.
protocol7b
added a comment - We're seeing similar problems in the ASF Hudson environment. Archiving frequently make up 90% of the build time for projects. Is the currently any work ongoing in this area? Is there anything we could assist that would help in debugging the problems we're seeing?
protocol7b
added a comment - We're seeing similar problems in the ASF Hudson environment. Archiving frequently make up 90% of the build time for projects. Is the currently any work ongoing in this area? Is there anything we could assist that would help in debugging the problems we're seeing? 
orekutin
added a comment - I diagnosed this issue some and I see a stack trace being sent with every 8K data chunk for these transfers. There's some 12 packets being sent per 8K chunk of data.
orekutin
added a comment - I diagnosed this issue some and I see a stack trace being sent with every 8K data chunk for these transfers. There's some 12 packets being sent per 8K chunk of data.
Created an attachment (id=754)
pom that simply jars hudson.war