The setting "Xvfb specific displayname" should select a random display from 1..100 if not specified. However, it can also select display 0, which could make the job fail if X is already running on :0.

From log output:
Xvfb starting$ Xvfb :0 -screen 0 1024x768x24 -fbdir /var/lib/jenkins/2012-10-24_14-31-395902912578711360537xvfb