In my case, I was running with -Xmx1500m, but CPU was (after a period of normal running, during which memory was leaked) hitting 100% with the GC running all the time and eventually threads were being killed due to the GC running too much (without much success).
In the java-heap post-mortem, there were loads of 8meg byte buffers - about a gigabyte's worth. I would have expected that if the garbage collector was able to collect them, it would have collected all of the ones it could have done and hence reduced this down to one or two at most (at the very most, I'd expect 1 per project, if all projects were polling at the same time the heap dump was made).
My guess is that these buffers would have only been disposed of once the slave connection had died, which is why your setup didn't suffer from this issue (as you take your slaves offline when they're not needed). On my setup, I keep the slaves online all the time (except when they reboot), so the buffers can build up over time (until they occupy enough memory to flatline the GC).
As mentioned above (22/Aug/11 10:49am), I've found that polling on the master (only) cures the problem (as well as preventing spurious builds caused by slaves being offline), which is why I think that it's a Jenkins-core problem and not an accurev-plugin bug (the plugin isn't obeying best practise of processing the command output on the slave, but that should just result in lower efficiency, not a memory leak).