When I was setting up a 'blog as a student of a digital media literacy program, I noticed that Vultr is offering a promotion. Presently, Vultr is showing a $250 initial credit for new accounts.
As one intermediate concern, this credit appears to have a shelf-life. I haven't discovered any documentation about when it expires. Mine has expired, after about a month or so. Happily, I'd only set up a really basic server for initial testing. It's working out so far, for the most part.
Compared to Digital Ocean: My FreeBSD instance at Digital Ocean stopped functioning, after a certain point, insofar as for cloud-init integration with rc.conf on the instance at DO. So, maybe it's not a fair comparison. Vultr works with full functionality. With Digital Ocean, I have to manually configure the IP and IPv6 addresses to be able to access the instance remotely. Having transitioned to Vultr, I haven't tried reinitializing the snapshot at DO or installing a new image, to see if it may reset anything in the cloud-init support (my last hope for making use of Digital Ocean's support at GitHub Student Developer Packs). So, there's Vultr though.
I've been able to configure a working strongSwan server with my FreeBSD 13.1 root-on-ZFS image at Vultr. This weekend, I believe I've figured out most of the firewall config. It's working out, to a certain extent. The following is my ipfw config in rc.conf:
As mentioned in the documentation, the firewall log can be accessed with
For configuring the firewall, the following is the
The filtering for ipsec-related protocols here is based on IPFW & NAT for strongswan VPN
Without trying to create a walkthrough here, for creating a certificate authority for strongSwan, and how to configure swanctl.conf for the certificate authority, the following is my server-side updown script for swanctl.conf on FreeBSD. This was referenced partly on a blog article from genneko, Route-based VPN with FreeBSD-11.1's IPsec VTI
Except for the special handling for
When used as the 'updown' script in swanctl.conf on the VPN server, this script will create a gif interface for each strongswan client, when each SA is initiated or rekeyed. This script can be used on FreeBSD strongSwan servers and clients. The strongSwan route-based documentation offers an example for Linux.
The bridge handling allows for communication among peers within the VPN private subnet. This may not typically be required on the VPN peers, except if there was a sort of mesh configuration in the design of the VPN .
This configuration results in being able to ping each strongswan client from the server, using the client's private address on the VPN. With a similar configuration on each strongswan client, I can also ping the server's private VPN address from the client. After a subsequent update in this config - and using the bridge interface on the VPN server - it's also possible to connect among VPN clients in the same effective VPN subnet.
This configuration has been sufficient for forwarding any local Squid proxy connections to a Squid instance on the server. I've not tried using NFS over the VPN link. ZFS send/ZFS receive works, though with bandwidth limitations due to my networking topology. To some extent, I've been able configure a private VPN with Vultr.
Between my FreeBSD and Linux client machines and this configuration on the server, however, it's not been sufficient for using the server as a normal gateway.
I've tried the following on each client, using Linux (armbian) and a local FreeBSD 13.1 built from stable/13 - client-side workflow:
Using gif interfaces on the server side, I can at least ping the VPN server from the client, and I can ping the clients from the server, over each VPN client link. It seemed ping was not working here, when I was using ipsec interfaces instead of gif interfaces, in the updown script for strongSwan.
Once the VPN client is initialized, I can then send any Squid proxy requests, Mosh connections, or probably any Socks proxy connections over the VPN link. It seems that it's working out as a private VPN network, at least.
With this configuration, the Android strongSwan client hasn't appeared to be of much use except for some limited testing. This may be due to the failing gateway support on the VPN server side, at present. The app has not been sending any Android data over the VPN, as verified with `swanctl --list-sas` on the server side. At one point, however, I'd tried connecting to one of the private VPN IPs from a shell under the Termux app on the Android phone, and that worked out. Albeit just a guess, maybe it would start sending normal network data over the VPN link, if the gateway support was working out on the VPN server...
Using the Android built-in VPN client, I haven't been able to get the Android built-in VPN client to complete the connection with the server here. Maybe this too could be related to the failing gateway support on the server - both of these clients may be trying to connect to some third-party service after the SA negotation from strongSwan.
For proxy support though, and generally for private networking with the strongSwan VPN link over FreeBSD, it works...
To create the server and client certificates for StrongSwan, I used the PKI tools from strongswan and a Makefile, with manual certificate distribution. I managed to figure out the auth configuration and remote_ts/local_ts specification after some trial and error. It might be a reproducible configuration, to some extent.
For compatibility with the Android client, I'm using mschap v2 auth throughout, in the VPN side. I'd been hoping that the strongSwan Android client might eventually start to send data over the VPN link after working out the NAT configuration on the server side, but there's been no change at that so far. At one point in this process - using tcpdump for the NAT'd traffic, albeit then under the pf firewall - it seemed that something on the phone is accessing a specific off-site URL - or maybe it's not using HTTP, but some third-party service - after SA initiation. I haven't tried to debug this shadowy bit of stuff any further. Maybe there's any clue about it, in the strongSwan Android client's source code, if it's not something introduced from the carrier or from the Android platform on that specific model of Android device.
HTH. Maybe anyone has a guess to why the VPN gateway part isn't working out, here?
Update: Initially, I was able to ping the VPN gateway address from VPN clients, but could not reach other VPN clients. After adjusting the
The
Configuration excerpt from swanctl.conf, for the VPN server:
After adjusting the configuration on each strongSwan client to use that
After adding a bridge interface on the server then adding each gif interface to that bridge, with that bridge being assigned with a single address in the same subnet as shown in in
The server's strongswan pool was configured to assign addresses above 10.10.0.1/24
Excerpt from swanctl.conf on the server peer:
The subnet for the cloud-00 pool may seem to correspond to the netblock in the
At least it's a complete private VPN now. Albeit, it's still not working out if trying to use the VPN server as a gateway. If that may be a matter of the server-side routing/nat configuration, maybe it would "Just work" with strongSwan connecting to a third-party VPN service, e.g protonvpn or other.
As one intermediate concern, this credit appears to have a shelf-life. I haven't discovered any documentation about when it expires. Mine has expired, after about a month or so. Happily, I'd only set up a really basic server for initial testing. It's working out so far, for the most part.
Compared to Digital Ocean: My FreeBSD instance at Digital Ocean stopped functioning, after a certain point, insofar as for cloud-init integration with rc.conf on the instance at DO. So, maybe it's not a fair comparison. Vultr works with full functionality. With Digital Ocean, I have to manually configure the IP and IPv6 addresses to be able to access the instance remotely. Having transitioned to Vultr, I haven't tried reinitializing the snapshot at DO or installing a new image, to see if it may reset anything in the cloud-init support (my last hope for making use of Digital Ocean's support at GitHub Student Developer Packs). So, there's Vultr though.
I've been able to configure a working strongSwan server with my FreeBSD 13.1 root-on-ZFS image at Vultr. This weekend, I believe I've figured out most of the firewall config. It's working out, to a certain extent. The following is my ipfw config in rc.conf:
Code:
gateway_enable="YES"
ipv6_gateway_enable="YES"
forward_sourceroute="YES"
firewall_enable="YES"
firewall_type="/etc/ipfw.sh"
firewall_nat_enable="YES"
firewall_nat64_enable="YES"
firewall_nat_interface="vtnet0"
firewall_logif="YES"As mentioned in the documentation, the firewall log can be accessed with
tcpdump -t -n -i ipfw0 subsequent of ifconfig ipfw0 create. After enabling `firewall_logif` in rc.conf, the ipfw0 interface would normally be configured at boot. It can be created without rebooting, using ifconfig ipfw0 create directly.For configuring the firewall, the following is the
/etc/ipfw.sh I'm using, after the documentation for kernel-based NAT, in the FreeBSD handbook. For purpose of testing, this script can be run directly with '-d', such as to enable debugging in the script and logging for NAT. A public interface name may also be provided to the script, after any `-d`. At Vultr, the default public interface may generally be `vtnet0` on FreeBSD. So, that's the default public interface name here.The filtering for ipsec-related protocols here is based on IPFW & NAT for strongswan VPN
Code:
#!/bin/sh
LOG_NAT=""
case $1 in
-d)
shift
set -x
LOG_NAT=log
;;
esac
set -e
## IPFW config
PIF=${1:-vtnet0}
## used below, with the server-side updown scripting for strongswan
VPNBR=bridge100
INIF="gif* ${VPNBR}"
OK_OUT_UDP="53,67"
OK_OUT_TCP="22,25,37,53,80,443,110"
case $(sysrc -qn strongswan_enable) in
[Yy][Ee][Ss])
OK_OUT_UDP="${OK_OUT_UDP},500,4500"
;;
esac
## IPFW macros
ADD='ipfw -q add'
NAT_IDX=5000
GOTO_NAT="skipto ${NAT_IDX}"
KS='keep-state'
## Internal config
INIF_IDX=5
## IPFW pre-config
## enable dynamic state persistence
sysctl -qn net.inet.ip.fw.dyn_keep_states=1 >/dev/null
## see docs
sysctl -qn net.inet.tcp.tso=0 >/dev/null
ipfw -q -f flush
## configure for stateful NAT
ipfw disable one_pass
## configure outbound NAT
ipfw -q nat 1 config if ${PIF} same_ports unreg_only reset
for IF in ${INIF}; do
$ADD ${INIF_IDX} allow all from any to any via ${IF}
INIF_IDX=$((INIF_IDX + 1))
done
LO_ALLOW_IDX=$((INIF_IDX * 10))
$ADD ${LO_ALLOW_IDX} allow all from any to any via 'lo*'
$ADD $((LO_ALLOW_IDX + 5)) reass all from any to any in ## reassemble inbound
## inbound NAT
$ADD $((LO_ALLOW_IDX + 50)) nat 1 ip from any to any in via ${PIF}
$ADD $((LO_ALLOW_IDX + 100)) check-state
PIF_IDX=$((LO_ALLOW_IDX + 150))
pif_nat() {
$ADD ${PIF_IDX} ${GOTO_NAT} $@
PIF_IDX=$((PIF_IDX + 5))
}
for S in ${OK_OUT_UDP}; do
pif_nat udp from any to any ${S} out via ${PIF} ${KS}
done
for S in ${OK_OUT_TCP}; do
pif_nat tcp from any to any ${S} out via ${PIF} setup ${KS}
done
pif_nat icmp from any to any out via ${PIF} ${KS}
$ADD $((PIF_IDX+5)) accept tcp from any to any ${OK_OUT_TCP} in via ${PIF} setup ${KS}
$ADD $((PIF_IDX+5)) accept udp from any to any ${OK_OUT_UDP} in via ${PIF} ${KS}
## mosh in on PIF
$ADD $((PIF_IDX+6)) accept udp from any to any 60000-61000 in via ${PIF} ${KS}
WAN_IDX=$((PIF_IDX+10))
$ADD ${WAN_IDX} allow esp from any to any
$ADD ${WAN_IDX} allow ah from any to any
$ADD ${WAN_IDX} allow ipencap from any to any
$ADD ${WAN_IDX} allow udp from any to me in recv ${PIF} frag
DENY_IDX=$((PIF_IDX + 50))
$ADD ${DENY_IDX} deny log all from any to any
$ADD ${NAT_IDX} nat 1 ${LOG_NAT} ip from any to any out via ${PIF}
$ADD $((NAT_IDX + 5)) allow ip from any to anyWithout trying to create a walkthrough here, for creating a certificate authority for strongSwan, and how to configure swanctl.conf for the certificate authority, the following is my server-side updown script for swanctl.conf on FreeBSD. This was referenced partly on a blog article from genneko, Route-based VPN with FreeBSD-11.1's IPsec VTI
Code:
#!/bin/sh
## updown script - enabled with swanctl.conf
## docs:
## https://docs.strongswan.org/docs/5.9/plugins/updown.html
IPSEC_IF=$(printf "gif1%02d" ${PLUTO_REQID})
BRIDGE_IF=bridge100
if ! ifconfig ${BRIDGE_IF} 1>&2 2>/dev/null; then
## the address assigned to the bridge interface may vary by strongswan configuration,
## and can be configured in rc.conf
ifconfig ${BRIDGE_IF} create
fi
case ${PLUTO_VERB} in
up-client)
if ! ifconfig ${IPSEC_IF} 1>&2 2>/dev/null; then
ifconfig ${IPSEC_IF} create
fi
ifconfig ${IPSEC_IF} descr "${PLUTO_CONNECTION}/${PLUTO_PEER_ID}#${PLUTO_REQID}(${PLUTO_UNIQUEID})" tunnel ${PLUTO_ME} ${PLUTO_PEER}
ifconfig ${IPSEC_IF} inet ${PLUTO_MY_CLIENT} ${PLUTO_PEER_CLIENT%/*} up
route add ${PLUTO_PEER_CLIENT} ${PLUTO_PEER_CLIENT%/*}
ifconfig ${BRIDGE_IF} addm ${IPSEC_IF} 1>&2 2>/dev/null
;;
down-client)
if ifconfig ${IPSEC_IF} 1>&2 2>/dev/null; then
ifconfig ${IPSEC_IF} destroy
fi
if ! route del ${PLUTO_PEER_CLIENT} ${PLUTO_PEER_CLIENT%/*} 1>&2 2>/dev/null; then true; fi
;;
esacExcept for the special handling for
BRIDGE_IF, otherwise this server-side script can also be used for the VPN client peers running FreeBSD.When used as the 'updown' script in swanctl.conf on the VPN server, this script will create a gif interface for each strongswan client, when each SA is initiated or rekeyed. This script can be used on FreeBSD strongSwan servers and clients. The strongSwan route-based documentation offers an example for Linux.
The bridge handling allows for communication among peers within the VPN private subnet. This may not typically be required on the VPN peers, except if there was a sort of mesh configuration in the design of the VPN .
This configuration results in being able to ping each strongswan client from the server, using the client's private address on the VPN. With a similar configuration on each strongswan client, I can also ping the server's private VPN address from the client. After a subsequent update in this config - and using the bridge interface on the VPN server - it's also possible to connect among VPN clients in the same effective VPN subnet.
This configuration has been sufficient for forwarding any local Squid proxy connections to a Squid instance on the server. I've not tried using NFS over the VPN link. ZFS send/ZFS receive works, though with bandwidth limitations due to my networking topology. To some extent, I've been able configure a private VPN with Vultr.
Between my FreeBSD and Linux client machines and this configuration on the server, however, it's not been sufficient for using the server as a normal gateway.
I've tried the following on each client, using Linux (armbian) and a local FreeBSD 13.1 built from stable/13 - client-side workflow:
- Initiate the strongswan connection:
swanctl --initiate --child my-conn-child-namefor my-conn-child-name being a connection 'child' name configured in local swanctl.conf - Ping the VPN address for the client, to ensure the VPN link is available. On FreeBSD, this VPN address will be configured on a gif interface. On Linux, I'm using vti/ipsec interfaces - the armbian client does not appear to have support for Linux XFRM interfaces.
- Add a static route for the public IPv4 address of the StrongSwan server, routing over the non-VPN gateway link. This may serve to ensure that any later '--rekey` requests, as well as Mosh and general SSH traffic can reach the server over its public interface, regardless of the next change in routing.
- Remove any other default routing
- Configure the default route to send all traffic over the VPN address for the client. On Linux, I've tried configuring this by using the client's VPN address for the routing - it won't accept the server's VPN address. On FreeBSD, I've tried this configuration with each of the VPN's peer's own 'local_ts' address, or the address serving as the ' remote_ts' address (may be a client IP addreses or a subnet address) negotiated by strongswan, or the address 10.10.0.1 which is now assigned to the server's bridge interface for VPN peers. None of these approaches seems to work, or maybe there's something else missing.
- Try to fetch the example.com main index page with wget, using the default route over VPN
Using gif interfaces on the server side, I can at least ping the VPN server from the client, and I can ping the clients from the server, over each VPN client link. It seemed ping was not working here, when I was using ipsec interfaces instead of gif interfaces, in the updown script for strongSwan.
Once the VPN client is initialized, I can then send any Squid proxy requests, Mosh connections, or probably any Socks proxy connections over the VPN link. It seems that it's working out as a private VPN network, at least.
With this configuration, the Android strongSwan client hasn't appeared to be of much use except for some limited testing. This may be due to the failing gateway support on the VPN server side, at present. The app has not been sending any Android data over the VPN, as verified with `swanctl --list-sas` on the server side. At one point, however, I'd tried connecting to one of the private VPN IPs from a shell under the Termux app on the Android phone, and that worked out. Albeit just a guess, maybe it would start sending normal network data over the VPN link, if the gateway support was working out on the VPN server...
Using the Android built-in VPN client, I haven't been able to get the Android built-in VPN client to complete the connection with the server here. Maybe this too could be related to the failing gateway support on the server - both of these clients may be trying to connect to some third-party service after the SA negotation from strongSwan.
For proxy support though, and generally for private networking with the strongSwan VPN link over FreeBSD, it works...
To create the server and client certificates for StrongSwan, I used the PKI tools from strongswan and a Makefile, with manual certificate distribution. I managed to figure out the auth configuration and remote_ts/local_ts specification after some trial and error. It might be a reproducible configuration, to some extent.
For compatibility with the Android client, I'm using mschap v2 auth throughout, in the VPN side. I'd been hoping that the strongSwan Android client might eventually start to send data over the VPN link after working out the NAT configuration on the server side, but there's been no change at that so far. At one point in this process - using tcpdump for the NAT'd traffic, albeit then under the pf firewall - it seemed that something on the phone is accessing a specific off-site URL - or maybe it's not using HTTP, but some third-party service - after SA initiation. I haven't tried to debug this shadowy bit of stuff any further. Maybe there's any clue about it, in the strongSwan Android client's source code, if it's not something introduced from the carrier or from the Android platform on that specific model of Android device.
HTH. Maybe anyone has a guess to why the VPN gateway part isn't working out, here?
Update: Initially, I was able to ping the VPN gateway address from VPN clients, but could not reach other VPN clients. After adjusting the
local_ts in swanctl.conf on the server side, I can now ping the Linux client from a FreeBSD client on the VPN. My previous local_ts was 10.10.0.1, which worked out for peer-to-server connections. Using local_ts = 10.10.0.0/24 i.e a network address, and a single gateway address with a bridge interface on the server, then using an updown script for each of Linux and FreeBSD on the client side, now I can ping a Linux client B from a FreeBSD client A on the same VPN, and also connect to services in the VPN subnet on the server.The
local_ts on the server side appears to correspond to the address pool configuration in swanctl.conf. It should also correspond to the remote_ts on the client side, for clients accepting this configuration field. (There's no such configuration field in the strongSwan client on Android)Configuration excerpt from swanctl.conf, for the VPN server:
Code:
children {
net-myservername {
## https://wiki.strongswan.org/projects/strongswan/wiki/Fromipsecconf
dpd_action=start
start_action = start
ipcomp = yes
local_ts = 10.10.0.0/24
}
}After adjusting the configuration on each strongSwan client to use that
local_ts as the client's remote_ts, I can then ping across that subnet, from each strongSwan client.After adding a bridge interface on the server then adding each gif interface to that bridge, with that bridge being assigned with a single address in the same subnet as shown in in
local_ts there, then I can also ping the server from the clients. I'm using 10.10.0.1/24 for the server, here. This address is now being assigned to a bridge interface, with each gif client interface added to that bridge.The server's strongswan pool was configured to assign addresses above 10.10.0.1/24
Excerpt from swanctl.conf on the server peer:
Code:
pools {
cloud-00 {
addrs = 10.10.0.10/24
}
cloud6-00 {
addrs= fd03:7b77:73fd:1:50:50:50:0/120
}
}The subnet for the cloud-00 pool may seem to correspond to the netblock in the
local_ts entry for this VPN server.At least it's a complete private VPN now. Albeit, it's still not working out if trying to use the VPN server as a gateway. If that may be a matter of the server-side routing/nat configuration, maybe it would "Just work" with strongSwan connecting to a third-party VPN service, e.g protonvpn or other.