I don't kld load at boot graphics components.... for debugging reasons.
I start my systems under console with a maximum compatibility and stability in mind... so I stay by default in the VGA text mode.
In vga text mode, no need of nvidia.ko and nvidia-modeset, this is the maximum compatibility mode.
If I run Xorg, launching xorg should automatically launch graphic kernel modules dependencies.
if I decide to switch the VT console in high resolution, so I kld load manually radeonkms.ko, i915kms.ko or nvidia-modset according to the hardware.
The advantage is.... if you install an update of nvidia-driver and this new update causes issues like kernel panic, the system will systematically crash at restart, you need to go to single user mode to edit rc.conf and/or loader.conf to deactivate temporarily theses modules.