Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3799

Slave-to-master copies can be extremely slow

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Major
    • Resolution: Duplicate
    • core
    • None
    • Platform: All, OS: All

    Attachments

      Issue Links

        Activity

          I was approached by someone in the IRC channel that said he's going to take a
          look at it, and after I said thank you to him and he left, I realized that I was
          actually supposed to fix this problem for my colleagues.

          So I went ahead and take a look anyway.

          The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo()
          was flushing buffers too often, and that was causing the unnecessary network
          traffic. I also removed the recording of where Command objects are created for
          ProxyOutputStream.Chunk to cut the overhead.

          This fix will be in 1.311.

          kohsuke Kohsuke Kawaguchi added a comment - I was approached by someone in the IRC channel that said he's going to take a look at it, and after I said thank you to him and he left, I realized that I was actually supposed to fix this problem for my colleagues. So I went ahead and take a look anyway. The root cause was that the TarOutputStream used in FilePath.copyRecursiveTo() was flushing buffers too often, and that was causing the unnecessary network traffic. I also removed the recording of where Command objects are created for ProxyOutputStream.Chunk to cut the overhead. This fix will be in 1.311.

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/main/java/hudson/FilePath.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java
          trunk/www/changelog.html
          http://fisheye4.cenqua.com/changelog/hudson/?cs=18816
          Log:
          [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.

          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/FilePath.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/Command.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=18816 Log: [FIXED JENKINS-3799] Fix for performance problem in FilePath.copyRecursiveTo() over the network.
          abayer Andrew Bayer added a comment -

          I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the
          specific problem I'm seeing - it's showing up for Maven artifacts, which are
          getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().

          abayer Andrew Bayer added a comment - I'll try this out once 1.311 comes out, but I'm not 100% sure this will fix the specific problem I'm seeing - it's showing up for Maven artifacts, which are getting copied using FilePath.copyTo(), not FilePath.copyRecursiveTo().
          abayer Andrew Bayer added a comment -

          This is still going on.

          abayer Andrew Bayer added a comment - This is still going on.
          zynasis zynasis added a comment -

          Having the same issue. Using v1.366

          zynasis zynasis added a comment - Having the same issue. Using v1.366
          asimmons Aaron Simmons added a comment -

          This problem is still occurring as of 1.403

          asimmons Aaron Simmons added a comment - This problem is still occurring as of 1.403
          jglick Jesse Glick added a comment -

          Probably improved by JENKINS-7813 fixes.

          jglick Jesse Glick added a comment - Probably improved by JENKINS-7813 fixes.

          People

            Unassigned Unassigned
            abayer Andrew Bayer
            Votes:
            13 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: