Thorsten, I've uploaded a new version on Dropbox. I think that I've finally cracked it, although I'm not all too happy about the implementation: Xvfb is started with matrix build, subsequent axis jobs are run and counted, when the matrix job finishes total that count is decreased and if it gets to zero Xvfb is stopped. You see, I could not find a way to get notified that all matrix jobs are done, so counting is the best thing I could think of. Anyways I realized that I needed to test better as well, and it seems to workout nice in my environment.
Thank you for your patience, I hope that this will finally fix it. If you can test in your environment once again and confirm that I would much appreciate it.