I have "Start Xvfb before the build, and shut it down after." enabled.
Without "Let Xvfb choose display name" enabled, the first execution of a job would run and leave Xvfb running so that the next execution of the build (within a few minutes) would fail:

Xvfb starting$ Xvfb :1 -screen 0 1024x768x24 -fbdir /xvfb-15-243204126288424340.fbdir
(EE)
Fatal server error:
(EE) Server is already active for display 1
If this server is no longer running, remove /tmp/.X1-lock
and start again.
(EE)
unlink: No such file or directory
unlink failed, Invalid argument
ERROR: Xvfb failed to start, consult the lines above for errors
Archiving artifacts
Build did not succeed and the project is configured to only push after a successful build, so no pushing will occur.
Finished: FAILURE

Running "ps ax" on the slave would show the process running.

If I happen to wait a few minutes (not sure how many), then it will not fail, but only the previous Xvfb process is running (a new one is not started).

Someone suggested me turning on "Let Xvfb choose display name". Now a new process is started for each build and left running (and accumulating).

I also tried enabling "Shutdown Xvfb with whole job, not just with the main build action" and that didn't help.

I enabled "Log Xvfb output", but I don't see any extra logging at shutdown. Is there a way to diagnose why Xvfb is not stopping? System logs to look at?