One of the things I've notice when installing X dependent ports and packages is that ports/pkg_add will install all the X libs, and such but not the X server.. for cases where perhaps the system is headless to be connected via VNC or perhaps to run the
Xvfb environment, which is for intents and purposes an X server, minus the monitor and fancy graphics drivers. So it is typically up to the end user to explicitly specify which of the 3 options they wish to run, be it,
Xorg,
Xvfb, or
VNC. This is what helps FreeBSD remain the best choice for everyone, no matter the desired solution to whatever need is present.

I should note, I use
Xvfb to run an automated web site snapshot utility using
cutycap. A cron fires and starts the
Xvfb server, loads
cutycap and takes snapshots of a number of sites, and flash even works. This is run from inside a
jail on a headless system, so it would be extremely impractical to have the entire
Xorg system installed, when I don't need it. Thanks to the Ports developers for keeping things such as that in mind </rant>