As a matter of fact,
setkey(8) is the tool for manual configuration and inspection of the Security Association Database (
SAD) and the Security Policy Database (
SPD) in the Kernel. Look at the
Examples section of the linked man file, for getting an idea. That said, manually setting up and maintaining the SAD/SPD may be quite cumbersome, and this becomes even worse if it comes to Authentication and Key Exchange.
... we could say ESP, Blah, Blah, Blah.
The 3 trailing Blah's in your recent message are in IPsec terms actually AH, SA, ISAKMP (IKE/IKEv2), and if you really want to set up everything manually, then you need to understand the concepts before, otherwise you will fail.
People, like me, who don't need to work out every nut, bolt and screw for any kind of satisfaction, utilize for establishing IPsec links either of
security/ipsec-tools or
security/strongSwan, the latter of which is more modern and better maintained.
BTW: I am not a Windows guy, however, a quick review of
Windows Firewall with Advanced Security and IPsec reveals to me that said Firewall does neither establish any IPsec, nor takes care of Authentication and Key Exchange. For this you need other facilities of Windows. The document tells me that the
WFASI knows to apply its rules also on IPsec traffic, as
IPFW knows on FreeBSD, however, no *NIX aware person would expect
IPFW to establish IPsec connections.