Incidentally, I was facing the same problem at the same time. Also wanted to run a headless server with Xvfb and VNC, not on RPi though. I found a working solution doing the following steps...
apt-get install lubuntu-core xvfb x11vnc
I figured out that all configuration options of lightdm.conf are documented in /usr/share/doc/lightdm/lightdm.conf.gz. So have a look at them by issuing the following command.
zcat /usr/share/doc/lightdm/lightdm.conf.gz
Obviously, as you reported, lightdm tries to instantiate its own X server and passes some arguments that Xvfb can't handle. First step to work around this is adding a line 'xserver-command' to lightdm's configuration file /etc/lightdm/lightdm.conf (it defaults to xserver-command=X).
[SeatDefaults] greeter-session=lightdm-gtk-greeter user-session=Lubuntu xserver-command=/etc/X11/xinit/xserverrc
After that, I modified /etc/X11/xinit/xserverrc so as to start Xvfb instead of a real X server (note that I commented out the original X exec line that passes the command line arguments on to X). Adding an exec line instead that runs Xvfb was enough to get lightdm working with Xvfb.
#!/bin/sh #exec /usr/bin/X -nolisten tcp "$@" exec Xvfb :0 -screen 0 1024x768x24
This seems to me like a convenient method of wrapping the lightdm X command in a suitable wrapper script that is already present on the (L)Ubuntu default installation.
Finally, I use VNC after ssh'ing into the system, forwarding the VNC port and connecting to the forwarded port on localhost with a VNC client (in my case Mac OS screen sharing).
ssh -L 5900:localhost:5900 user@machine 'x11vnc -localhost -display :0 -many'