Although I can't prove it by numbers, I also have the feeling that most of the time when a build happened with jgit jenkins dies with an OutOfMemory error.

How "large" is your repository ? Do you know a public one I could use to run some tests with a profiler ?

The problem repository is 2.8 GB. I don't know of any public repository that would be that large, though I suspect you might see the same problem with the Linux kernel repository at git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git . I've heard it has reached hundreds of megabytes now.

If the problem does not duplicate with the Linux kernel repo, you could probably duplicate the problem with a repository you create yourself using large files. My repo includes a checked in copy of Microsoft SQL Server 2008 Express and a bunch of other large binaries.

I'm not sure it is worth using a public repository for the replication of the bug, since the network transfer will waste a lot of your time. Wouldn't it be better to choose a collection of large files, submit them to a local git repo, then try to clone that git repo locally?

I can reproduce this issue (at least, OOME) using http://github.com/torvalds/linux.git

Is this something we should even attempt to solve in Jenkins? IMO it would suffice to document (e.g. in a blurb in the JGit tool configuration section) that JGit is known to not handle extremely large repositories as well as the native Git client. If and when the memory usage patterns (or whatever the root problem is) are fixed upstream, we will integrate the updated JGit client as a matter of course. In the meantime, people with such large repositories can continue to use the native client as before.