There's one exception.
If using nvidia GPU without any of graphics/nvidia-drm-[510|515|61|66]-kmod, /etc/X11/xorg.conf (or /usr/local/etc/X11/xorg.conf.d/20-nvidia.conf in example 3 of Chapter 5.4.3 in the handbook, which anyway does not exist by default) is needed as xorg doesn't know how to configure it and attempts to use non-existent (on FreeBSD) nouveau driver, thus, fail.
This always applies for legacy drivers (470 series and older), as graphics/nvidia-drm-[510|515|61|66]-kmod does not support legacy branches of drivers.