Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-1275

Artifact download slow (200kb/s)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Labels:
      None
    • Environment:
      Platform: All, OS: Linux
    • Similar Issues:

      Description

      Downloading an artifact published by a project is very slow - 200kb/s or so.
      While downloading, Hudson uses 100% CPU. In my case, the artifact has 168MB.

      Hudson v1.173 running on Linux lsts-cc 2.6.18-3-686 #1 SMP i686 GNU/Linux; CPU
      is an older Pentium(R) 4 CPU 1.80GHz

        Attachments

          Activity

          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          This is the main loop that serves files:

          byte[] buf = new byte[1024];
          int len;
          while((len=in.read(buf))>0)
          out.write(buf,0,len);

          Do you think 1K buffer is too small? Maybe that has something to do with this?
          Although that alone wouldn't explain 100% CPU usage.

          Please tell us what container you use. Perhaps that's doing something? Also, can
          you check the HTTP response header to see if it's doing the HTTP compression?
          Maybe that's the bottleneck?

          Show
          kohsuke Kohsuke Kawaguchi added a comment - This is the main loop that serves files: byte[] buf = new byte [1024] ; int len; while((len=in.read(buf))>0) out.write(buf,0,len); Do you think 1K buffer is too small? Maybe that has something to do with this? Although that alone wouldn't explain 100% CPU usage. Please tell us what container you use. Perhaps that's doing something? Also, can you check the HTTP response header to see if it's doing the HTTP compression? Maybe that's the bottleneck?
          Hide
          mvy2 mvy2 added a comment -

          I think 1k buffer is okay - I used to use 2k - 8k buffers but 1k buffer should
          not hog the cpu alone.
          I am running the Hudson with "java -jar hudson.war"

          exact command is:
          su $USER -c "java -jar $HUDSON_HOME/hudson.war --httpPort=$PORT >>$LOGFILE 2>&1" &

          AFAIK the http file transfer from Hudson is not using compression - here's
          output of wget -S:

          HTTP/1.0 200
          Server: Winstone Servlet Engine v0.9.9
          Last-Modified: Mon, 11 Feb 2008 23:43:17 GMT
          Expires: Mon, 11 Feb 2008 23:43:17 GMT
          Content-Type: application/octet-stream
          Content-Length: 176835050
          Connection: Keep-Alive
          Date: Tue, 12 Feb 2008 08:21:13 GMT
          X-Powered-By: Servlet/2.5 (Winstone/0.9.9)
          Length: 176,835,050 (169M) [application/octet-stream]

          I'll try to run the profiler, we might get more information.

          Show
          mvy2 mvy2 added a comment - I think 1k buffer is okay - I used to use 2k - 8k buffers but 1k buffer should not hog the cpu alone. I am running the Hudson with "java -jar hudson.war" exact command is: su $USER -c "java -jar $HUDSON_HOME/hudson.war --httpPort=$PORT >>$LOGFILE 2>&1" & AFAIK the http file transfer from Hudson is not using compression - here's output of wget -S: HTTP/1.0 200 Server: Winstone Servlet Engine v0.9.9 Last-Modified: Mon, 11 Feb 2008 23:43:17 GMT Expires: Mon, 11 Feb 2008 23:43:17 GMT Content-Type: application/octet-stream Content-Length: 176835050 Connection: Keep-Alive Date: Tue, 12 Feb 2008 08:21:13 GMT X-Powered-By: Servlet/2.5 (Winstone/0.9.9) Length: 176,835,050 (169M) [application/octet-stream] I'll try to run the profiler, we might get more information.
          Hide
          mvy2 mvy2 added a comment -

          Okay I just have run the Hudson with HPERF. I tried:
          java -agentlib:hprof=cpu=times -jar hudson.war
          After the Hudson launched, I immediately started to download the artifact. It
          downloaded approximately 1,1mb in 20 minutes (localhost connection, Intel(R)
          Core(TM)2 Quad CPU @ 2.40GHz). I'll attach the profiler output.

          Show
          mvy2 mvy2 added a comment - Okay I just have run the Hudson with HPERF. I tried: java -agentlib:hprof=cpu=times -jar hudson.war After the Hudson launched, I immediately started to download the artifact. It downloaded approximately 1,1mb in 20 minutes (localhost connection, Intel(R) Core(TM)2 Quad CPU @ 2.40GHz). I'll attach the profiler output.
          Hide
          mvy2 mvy2 added a comment -

          Created an attachment (id=176)
          HPROF -agentlib:hprof=cpu=times output

          Show
          mvy2 mvy2 added a comment - Created an attachment (id=176) HPROF -agentlib:hprof=cpu=times output
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          Thanks for the data. I found the root cause, which was in Winstone. See
          http://sourceforge.net/tracker/index.php?func=detail&aid=1903240&group_id=98922&atid=622497
          for more information.

          I'm patching this locally to fix the problem.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - Thanks for the data. I found the root cause, which was in Winstone. See http://sourceforge.net/tracker/index.php?func=detail&aid=1903240&group_id=98922&atid=622497 for more information. I'm patching this locally to fix the problem.
          Hide
          kohsuke Kohsuke Kawaguchi added a comment -

          Fixed in 1.186.

          Show
          kohsuke Kohsuke Kawaguchi added a comment - Fixed in 1.186.
          Hide
          mvy2 mvy2 added a comment -

          Thanks for resolving the issue! I'll wait for a release of 1.186 and mark the
          bug as VERIFIED later on.

          Show
          mvy2 mvy2 added a comment - Thanks for resolving the issue! I'll wait for a release of 1.186 and mark the bug as VERIFIED later on.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            mvy2 mvy2
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: