• Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • core
    • None
    • Platform: All, OS: All

        [JENKINS-3799] Slave-to-master copies can be extremely slow

        I was approached by someone in the IRC channel that said he's going to take a
        look at it, and after I said thank you to him and he left, I realized that I was
        actually supposed to fix this problem for my colleagues.

        So I went ahead and take a look anyway.

        The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo()
        was flushing buffers too often, and that was causing the unnecessary network
        traffic. I also removed the recording of where Command objects are created for
        ProxyOutputStream.Chunk to cut the overhead.

        This fix will be in 1.311.

        Kohsuke Kawaguchi added a comment - I was approached by someone in the IRC channel that said he's going to take a look at it, and after I said thank you to him and he left, I realized that I was actually supposed to fix this problem for my colleagues. So I went ahead and take a look anyway. The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo() was flushing buffers too often, and that was causing the unnecessary network traffic. I also removed the recording of where Command objects are created for ProxyOutputStream.Chunk to cut the overhead. This fix will be in 1.311.

        Code changed in hudson
        User: : kohsuke
        Path:
        trunk/hudson/main/core/src/main/java/hudson/FilePath.java
        trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java
        trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java
        trunk/www/changelog.html
        http://fisheye4.cenqua.com/changelog/hudson/?cs=18816
        Log:
        [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.

        SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/FilePath.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=18816 Log: [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.

        Andrew Bayer added a comment -

        I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the
        specific problem I'm seeing - it's showing up for Maven artifacts, which are
        getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().

        Andrew Bayer added a comment - I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the specific problem I'm seeing - it's showing up for Maven artifacts, which are getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().

        Andrew Bayer added a comment -

        This is still going on.

        Andrew Bayer added a comment - This is still going on.

        zynasis added a comment -

        Having the same issue. Using v1.366

        zynasis added a comment - Having the same issue. Using v1.366

        Aaron Simmons added a comment -

        This problem is still occurring as of 1.403

        Aaron Simmons added a comment - This problem is still occurring as of 1.403

        Jesse Glick added a comment -

        Probably improved by JENKINS-7813 fixes.

        Jesse Glick added a comment - Probably improved by JENKINS-7813 fixes.

          Unassigned Unassigned
          abayer Andrew Bayer
          Votes:
          13 Vote for this issue
          Watchers:
          14 Start watching this issue

            Created:
            Updated:
            Resolved: