Because non-port/package software tends to install executables, libraries and configuration files outside of [FILE]/usr/local/[/FILE], e.g. in [file]/bin/[/file], [file]/usr/bin/[/file], [FILE]/usr/lib/[/FILE], [file]/var/lib/[/FILE], [file]/etc/[/FILE], et cetera. That is strictly base system territory. Not only does this make it very hard to track down what gets installed where, but there's also usually not a proper way to deinstall external software, so remnants of it get left behind, and may interfere with later versions. Everything installed using a port or a package is traceable, maintainable, upgradeable, and deinstallable (because of [FILE]pkg-plist[/FILE], the ports Mk structure, and other mechanisms) using specialized tools ([FILE]pkg_*[/FILE] and port management tools).