Are you saying I need to 'git clone' the whole ports tree if I just want to build a single pkg?
If you want to find the source for a package, it takes doing some research.
The FreeBSD ports tree is merely the infrastructure to pull in source tarballs and compile them properly for your system. If you want to read the code of what's inside the tarball itself, you can look the package up on freshports, and see where the tarballs are published. This is where the ports infrastructure downloads the tarball from.
Another way to look at a package source tarball is in /usr/ports/distfiles.
But frankly, I'd recommend getting the whole ports tree in one go. This will save the headaches of hunting down deps later if what you have locally is outdated for the port you want.
Updating any given package - now that's a different bowl of fish, and invites dependency hell. However, that is something that all of Open Source Software is living with - BSD, Linux, you name it. Even Android is not great at updating single apps. Apple and Microsoft are closed source, and have their own ways of dealing with updates. However, they also have other, rather unrelated issues that drove me to FreeBSD...