The documentation says:
Xvfb screen changes the screen resolution and pixel depth, it needs to be in WxHxD format (i.e. 1600x1200x24). The default is 1024x768x24.

I just installed the xvfb-plugin, and took all the defaults, leaving the "screen" value blank. When running my Jenkins job, the first time, in the log I could see:

Xvfb starting$ Xvfb :1 -screen 0  -fbdir /var/lib/jenkins/xvfb-99-1021388786522495009.fbdir

and at the end of the job:

Xvfb stopping
Finished: SUCCESS 

Running the exact same job every subsequent time, the log say:

Xvfb starting$ Xvfb :1 -screen 0  -fbdir /var/lib/jenkins/xvfb-100-2665068639033469958.fbdir

ERROR: Xvfb failed to start, consult the lines above for errors

Looking at running jobs (using `ps -ef | grep -i xvfb`), I got nothing.

Trying this same command from the command line interface, I get:

$ Xvfb :1 -screen 0  -fbdir /var/lib/jenkins/xvfb-106-4356636144193454909.fbdir

Invalid screen configuration -fbdir for -screen 0

Finally I decided to specify something for the "screen" parameter - I used the supposed default: "1024x768x24" - and everything worked fine.