How do I route all Jail traffic through OpenVPN on FIB1

Rob J · Jan 8, 2021

Hello FreeBSD users, this is my fist post, I'm still learning so please be gentle

I am trying to setup a jail (using ioCage) so that all of its traffic goes through our VPN. I have already setup OpenVPN which connects to our VPN service without issue.

However the I seem unable

to successfully connect the jail to the VPN on FIB 1.

So far I have the following: (FreeBSD 12.2):

/boot/loader.conf

Code:

net.fibs=2

/etc/rc.conf

Code:

gateway_enable="YES"
pf_enable="YES"
defaultrouter="192.168.1.1"

/etc/rc.local

Code:

route add default 192.168.1.1 -fib 1
setfib -F 1 /usr/local/sbin/openvpn --config /root/myconfig.ovpn --daemon[/I]

This all seems to work OK, at least when I test it using:

# curl http://ipecho.net/plain

Which returns my public IP address

.

# setfib -F 1 curl http://ipecho.net/plain

This returns my VPNs public IP address

.

Next up I have created jail using ioCage:

# iocage create -n myjail -r LATEST vnet=on dhcp=on bpf=on allow_raw_sockets=on boot=on -T

myjail successfully created!
No default gateway found for ipv6.
* Starting myjail
+ Started OK
+ Using devfs_ruleset: 1002 (iocage generated default)
+ Configuring VNET OK
+ Using IP options: vnet
+ Starting services OK
+ Executing poststart OK
+ DHCP Address: 192.168.1.202/24

Next I set tun=1 and fib=1:

# iocage set allow_tun=1 myjail
# iocage set exec_fib=1 myjail

Restarted the jail:

# iocage restart myjail

Then accessed the jail using:

# iocage console myjail

The jail seems ok and is accessible on the network, however despite setting the jail fib=1 the jail does not seem utilize our VPN.

So from within our jail:

# curl http://ipecho.net/plain

Returns our normal public IP address

.

Again from within the jail:

Code:

# netstat -nr

Routing tables (fib: 1)

Internet:
Destination         Gateway         Flags         Netif         Expire
default                192.168.1.1     UGS         epair0b
127.0.0.1            link#1              UH            lo0
192.168.1.0/24   link#2              U               epair0b

Internet6:
Destination         Gateway         Flags         Netif         Expire
::/96                    ::1                   UGRS         lo0
::1                       link#1              UH             lo0
::ffff:0.0.0.0/96    ::1                   UGRS         lo0
fe80::/10             ::1                   UGRS         lo0
fe80::%lo0/64     link#1             U                 lo0
fe80::%epair0b/64     link#2      U                 epair0b
ff02::/16             ::1                   UGRS          lo0

Any help and advice to get this working would be greatly appreciated...

Thanks in advance.

Nemo1024 · Dec 8, 2021

This might be because of missing pf configuration. I have just spent several days trying to get a service running in a jail and have its in- and outbound traffic routed through VPN using FreeBSD 13-STABLE, OpenVPN and regular jail command (/etc/jail.conf). Finally I have made it work!

First, a list of all the resources that I used during my research. I first tried going the cloned interface way, but ended up with setting up a new FIB for the jail.

OpenVPN Client in a jail

Hello all, Host is FreeBSD 9.1/AMD64 I'd like to create a jail and have all traffic within said jail routed over an OpenVPN connection, but have traffic outside of the jail going straight out. I've tried it using straight up IP alias and that didn't work. I've found examples online of using...

forums.freebsd.org

OpenVPN in a FreeBSD Jail

FreeBSD and OpenVPN

www.carlomaiorano.me

Apply devfs rules in a jail

Hi all, I'm trying to setup cups to print to a usb-attached printer. I followed the instructions here: http://superuser.com/questions/673880/airprint-and-airplay-on-freenas to install cups in a jail on the FreeNAS machine. I've got cups working (if I manually chmod a+rw /dev/ulpt0), but I can't...

www.truenas.com

https://www.davd.io/freebsd-route-jail-traffic-through-vpn/

(This ended up being the main line of investigation)

How To Configure Packet Filter (PF) on FreeBSD 12.1 | DigitalOcean

PF is a renown firewall application that is maintained upstream by the security-driven OpenBSD project. It is more accurately expressed as a packet filtering…

www.digitalocean.com

Force a jail to use OpenVPN tunnel

So I have a few questions about how to best do this... I have an OpenVPN client on a host and I would like a jail on that host to use that tunnel for all non-local access. I also need to reach this jail from other hosts on the same subnet. The host/jail are behind a firewall, so no...

forums.freebsd.org

Routing a FreeBSD Jail through OpenVPN

I decided I wanted to concoct a solution where I could force all applications in a jail or jails through a VPN connection without affecting the internet connectivity of other daemons on the...

blog.feld.me

FIB on 10.1, adding gateway problem, bug?

I am running some daemons, including postfix on FIB 1 which has specific rules but the same gateway as the default routing table. In FreeBSD this was working just fine, but after the upgrade, the command setfib 1 route add default 192.168.1.1 fails with: route: writing to routing socket: Network...

forums.freebsd.org

PF - Safely share tun0 with eth0 (pf rule/s)

❔ I am connecting to a VPN and have implemented some basic rules to avoid leaks. I want to share the VPN-ized internet from my tun0 to my ethernet eth0 . I don't wish for eth0 to provide DHCP and eth0 has a static IP 10.10.10.1 I can enable gateway by using gateway_enable in rc.conf and sysctl...

forums.freebsd.org

Solved - Openvpn fails on system upgraded to 13.0 (Static route failure on fib)

I have just upgraded a server to 13.0 and now my vpn no longer works. It seems that I am no longer able to set a default route on fib 1 as the "Network is unreachable". I have the following in rc.conf, which used to work a treat, but no longer does: static_routes="vpn" route_vpn="default...

forums.freebsd.org

https://serverfault.com/questions/1069557/freebsd-13-pf-blocking-jail-traffic

https://forums.freebsd.org/threads/forward-packets-to-jail-with-pf.11362/

Solved - How to redirect public IP to jail IP

Hi guys, I am scratching my head in creating a pf rules to redirect my external public IP to my jails IPs. I have been given 8 IP addresses of which 5 are usable:96.205.75.242 /29 96.205.75.242 255.255.255.248 Subnet Address 96.205.75.243 255.255.255.248 Default Gateway 96.205.75.244...

forums.freebsd.org

The intended solution is pretty simple: a single service should run within the jail and listen to a range of ports. The jail should bypass the local network and talk to the outside world through VPN.

My setup is similar to OP's:

/boot/loader.conf

Code:

net.fibs=2

/etc/sysctl.conf

Code:

# Without this OpenVPN fails to initialise, even though the default route is set on FIB 1
net.add_addr_allfibs=1

This would be the error during OpenVPN initialization if the above is not set to 1 on FreeBSD 13. Maybe someone has a clue as to why and how to mitigate this:

Code:

2021-12-08 15:42:56 TUN/TAP device /dev/tun0 opened
2021-12-08 15:42:56 /sbin/ifconfig tun0 10.27.152.126 10.27.152.1 mtu 1500 netmask 255.255.255.0 up
2021-12-08 15:42:56 /sbin/route add -net 10.27.152.0 10.27.152.1 255.255.255.0
route: writing to routing socket: Network is unreachable
add net 10.27.152.0: gateway 10.27.152.1 fib 1: Network is unreachable
2021-12-08 15:42:56 ERROR: FreeBSD route add command failed: external program exited with error status: 1[/COLOR]

/etc/rc.conf

Code:

jail_enable="YES"
# I launch the jail later from a script after the VPN is up - thus empty list
jail_list=" "
gateway_enable="YES"
pf_enable="YES"
defaultrouter="192.168.1.1"

/etc/pf.conf

Code:

 set skip on { lo0 }
scrub in all
jail_ip = "192.168.1.20"
jail_service_range = "15000:15100"
nat pass on tun0 from $jail_ip to any -> ( tun0 )
rdr pass on tun0 proto tcp from any to any port $jail_service_range -> $jail_ip port $jail_service_range

** Run: service pf restart to effectuate the rules above **

/etc/jail.conf

Code:

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
mount.fstab = "/etc/fstab.$name";

servicejail {
     host.hostname = "servicejail";
     path = "/usr/local/jails/servicejail";
     interface = "em0";
     ip4.addr = 192.168.1.20;
     exec.fib=1;
}

/etc/rc.d/local

Code:

route add 192.168.1.0/24 -iface em0 -fib 1
route add default 192.168.1.1 -fib 1
setfib 1 /usr/local/sbin/openvpn --config /usr/local/etc/openvpn/config.ovpn --daemon

** Reboot **

** Run: jail -c servicejail once all the IPs are assigned and the DDNS services are updated.

Now, if I SSH into the jail and run nc -l 15000 to listen to the port in range, then do an external connectivity check to the VPN IP or DDNS name, I would see output from the nc(1) command. Running the same nc command on the host under both fib 0 and fib 1 does not give any result - the traffic is routed to the jail as expected.