Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-2001

SVN check out slow performance issue - REGRESSION 1.419 to 1.421

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • subversion-plugin
    • None
    • Platform: Sun, OS: Solaris

      The Subversion checkout in Hudson's continuous build is slow. Despite setting
      the checkout to run against a memory based Solaris file system, the checkout is
      still acting as slowly as to a regular filesystem. For example, the Subversion
      checkout for a memory based Solaris file system took just 1 minutes and 30
      seconds against 7 minutes and 9 seconds for Hudson's continuous build checkout
      into a memory based Solaris file system (more than 4.5 times slower). Another
      example, the Subversion checkout from NetBeans IDE took only 3 minutes and it
      was done into harddisk file system which faster for 2 times comparing with
      Hudson checkout into a memory based Solaris file system.

          [JENKINS-2001] SVN check out slow performance issue - REGRESSION 1.419 to 1.421

          ssburlg created issue -

          Does this involve master/slave set up? What is the SVN repository URL? Is this
          http, ssh, svn, or something else?

          Kohsuke Kawaguchi added a comment - Does this involve master/slave set up? What is the SVN repository URL? Is this http, ssh, svn, or something else?

          ssburlg added a comment -

          "Does this involve master/slave set up?"

          Nope.

          "What is the SVN repository URL?"

          http://toolssvn.sfbay.sun.com/svn/wow/trunk/platform

          Note: it's on SWAN

          "Is this http, ssh, svn, or something else?"

          Not a ssh as far as I know. Yes, it's http for accessing to SVN repository.

          Thank you Kohsuke for looking to this issue and should you have any questions,
          please let me know.

          ssburlg added a comment - "Does this involve master/slave set up?" Nope. "What is the SVN repository URL?" http://toolssvn.sfbay.sun.com/svn/wow/trunk/platform Note: it's on SWAN "Is this http, ssh, svn, or something else?" Not a ssh as far as I know. Yes, it's http for accessing to SVN repository. Thank you Kohsuke for looking to this issue and should you have any questions, please let me know.

          pgweiss added a comment -

          In my setup I have an svn+ssh URL. The command line client (SVN 1.5.1 running
          in Linux) takes 1 minutes 12 seconds to check out. Hudson takes about 15
          minutes.

          Is there a possible workaround? That is, can I get Hudson to not do the
          checkout at all, rather, just use its built in svn support for polling, and I
          will do the checkout as a build step?

          pgweiss added a comment - In my setup I have an svn+ssh URL. The command line client (SVN 1.5.1 running in Linux) takes 1 minutes 12 seconds to check out. Hudson takes about 15 minutes. Is there a possible workaround? That is, can I get Hudson to not do the checkout at all, rather, just use its built in svn support for polling, and I will do the checkout as a build step?

          pgweiss added a comment -

          This appears to be an svnkit issue rather than a hudson issue. Notice that
          svnkit takes an order of magnitude longer to check out than does the native
          svn.

          1. $SVN is an svn+ssh:// url

          [test] deng-pweis| ~/download/svnkit-1.2.0.4710/jsvn co $SVN/trunk/x
          ...
          Checked out revision 208171.
          ~/download/svnkit-1.2.0.4710/jsvn co $SVN/trunk/x 65.93s user 392.37s system
          63% cpu 12:05.85 total

          [test] deng-pweis| du -sh x
          1.3G qres

          [test] deng-pweis| rm -rf x

          [test] deng-pweis| time /usr/bin/svn co $SVN/trunk/x
          ...
          Checked out revision 208177.
          /usr/bin/svn co $SVN/trunk/x 8.24s user 6.06s system 18% cpu 1:17.06 total

          [test] deng-pweis| /usr/bin/svn --version
          svn, version 1.5.1 (r32289)
          compiled Jul 24 2008, 20:58:29

          Copyright (C) 2000-2008 CollabNet.
          Subversion is open source software, see http://subversion.tigris.org/
          This product includes software developed by CollabNet (http://www.Collab.Net/).

          The following repository access (RA) modules are available:

          • ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
          • handles 'http' scheme
          • handles 'https' scheme
          • ra_svn : Module for accessing a repository using the svn network protocol.
          • with Cyrus SASL authentication
          • handles 'svn' scheme
          • ra_local : Module for accessing a repository on local disk.
          • handles 'file' scheme
          • ra_serf : Module for accessing a repository via WebDAV protocol using serf.
          • handles 'http' scheme
          • handles 'https' scheme

          [test] deng-pweis| /usr/bin/java -version
          java version "1.6.0_07"
          Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
          Java HotSpot(TM) 64-Bit Server VM (build 10.0-b23, mixed mode)

          [test] deng-pweis| ~/download/svnkit-1.2.0.4710/jsvn --version
          jsvn, version 1.2.0 (r4710)

          Copyright (C) 2004-2007 TMate Software.
          SVNKit is open source (GPL) software, see http://svnkit.com/ for more
          information.
          SVNKit is pure Java (TM) version of Subversion, see http://
          subversion.tigris.org/

          The following repository access (RA) modules are available:

          • org.tmatesoft.svn.core.internal.io.dav : Module for accessing a repository
            via WebDAV protocol.
          • handles 'http' scheme
          • handles 'https' scheme
          • org.tmatesoft.svn.core.internal.io.svn: Module for accessing a repository
            using the svn network protocol.
          • handles 'svn' scheme
          • org.tmatesoft.svn.core.internal.io.fs: Module for accessing a repository on
            local disk.
          • handles 'file' scheme (only FSFS repositories are supported)

          pgweiss added a comment - This appears to be an svnkit issue rather than a hudson issue. Notice that svnkit takes an order of magnitude longer to check out than does the native svn. $SVN is an svn+ssh:// url [test] deng-pweis| ~/download/svnkit-1.2.0.4710/jsvn co $SVN/trunk/x ... Checked out revision 208171. ~/download/svnkit-1.2.0.4710/jsvn co $SVN/trunk/x 65.93s user 392.37s system 63% cpu 12:05.85 total [test] deng-pweis| du -sh x 1.3G qres [test] deng-pweis| rm -rf x [test] deng-pweis| time /usr/bin/svn co $SVN/trunk/x ... Checked out revision 208177. /usr/bin/svn co $SVN/trunk/x 8.24s user 6.06s system 18% cpu 1:17.06 total [test] deng-pweis| /usr/bin/svn --version svn, version 1.5.1 (r32289) compiled Jul 24 2008, 20:58:29 Copyright (C) 2000-2008 CollabNet. Subversion is open source software, see http://subversion.tigris.org/ This product includes software developed by CollabNet ( http://www.Collab.Net/ ). The following repository access (RA) modules are available: ra_neon : Module for accessing a repository via WebDAV protocol using Neon. handles 'http' scheme handles 'https' scheme ra_svn : Module for accessing a repository using the svn network protocol. with Cyrus SASL authentication handles 'svn' scheme ra_local : Module for accessing a repository on local disk. handles 'file' scheme ra_serf : Module for accessing a repository via WebDAV protocol using serf. handles 'http' scheme handles 'https' scheme [test] deng-pweis| /usr/bin/java -version java version "1.6.0_07" Java(TM) SE Runtime Environment (build 1.6.0_07-b06) Java HotSpot(TM) 64-Bit Server VM (build 10.0-b23, mixed mode) [test] deng-pweis| ~/download/svnkit-1.2.0.4710/jsvn --version jsvn, version 1.2.0 (r4710) Copyright (C) 2004-2007 TMate Software. SVNKit is open source (GPL) software, see http://svnkit.com/ for more information. SVNKit is pure Java (TM) version of Subversion, see http:// subversion.tigris.org/ The following repository access (RA) modules are available: org.tmatesoft.svn.core.internal.io.dav : Module for accessing a repository via WebDAV protocol. handles 'http' scheme handles 'https' scheme org.tmatesoft.svn.core.internal.io.svn: Module for accessing a repository using the svn network protocol. handles 'svn' scheme org.tmatesoft.svn.core.internal.io.fs: Module for accessing a repository on local disk. handles 'file' scheme (only FSFS repositories are supported)

          The original problem reported by ssburlg and the later addition by pgweiss are
          likely different as they involve different transports.

          In 1.260 I fixed a bug in SVNKit, which was breaking the use of persistent HTTP
          connections entirely. So hopefully this fixes the problem reported by ssburlg.

          Also in 1.260, we bumped up SVNKit to 1.2.0 release version, and one of the
          changelogs include svn+ssh performance improvement, which hopefully addresses
          the problem reported by pgweiss.

          If the new version won't show enough performance improvements, please feel free
          to reopen.

          Kohsuke Kawaguchi added a comment - The original problem reported by ssburlg and the later addition by pgweiss are likely different as they involve different transports. In 1.260 I fixed a bug in SVNKit, which was breaking the use of persistent HTTP connections entirely. So hopefully this fixes the problem reported by ssburlg. Also in 1.260, we bumped up SVNKit to 1.2.0 release version, and one of the changelogs include svn+ssh performance improvement, which hopefully addresses the problem reported by pgweiss. If the new version won't show enough performance improvements, please feel free to reopen.
          Kohsuke Kawaguchi made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

          ssburlg added a comment -

          The new version(1.260) doesn't showed performance improvements. Moreover, I
          experienced performance degradation, when I ran job which did just check out and
          it took 12 minutes, comparing with 9 minutes 22 second for version 1.230...

          So, I downgraded Hudson back to 1.230.

          ssburlg added a comment - The new version(1.260) doesn't showed performance improvements. Moreover, I experienced performance degradation, when I ran job which did just check out and it took 12 minutes, comparing with 9 minutes 22 second for version 1.230... So, I downgraded Hudson back to 1.230.
          ssburlg made changes -
          Resolution Original: Fixed [ 1 ]
          Status Original: Resolved [ 5 ] New: Reopened [ 4 ]

          bhoyt added a comment -

          I'm seeing this issue in 1.262. Subversion checkouts from a master are taking nearly 35 minutes, while
          the same checkout from the command line svn client on that master zips along in just a few minutes.
          This is a svn+ssh URL.

          bhoyt added a comment - I'm seeing this issue in 1.262. Subversion checkouts from a master are taking nearly 35 minutes, while the same checkout from the command line svn client on that master zips along in just a few minutes. This is a svn+ssh URL.

            Unassigned Unassigned
            ssburlg ssburlg
            Votes:
            7 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: