Questions about networking and (VNET) jails

[DeliQ]

I am trying to learn more about networking and jails. I have some questions though, and I was hoping you could help me.

1. Are there convincing reasons for running VNET jails, if the only objective is to sandbox services? Is it always worth it to have the overhead of multiple separate networking stacks?
2. A lot of VNET jail tutorials are based on if_bridge and if_epair devices. What advantage does bridging give you over just connecting hosts and jails with if_epair devices and routing between them?

Thank you

 

[DeliQ]

Regarding 2:
To elaborate: I understand there is value in adding your current physical egress interface to the bridge, to allow jails to be part of your current broadcast domain (network). Is that the only reason why you would want this configuration?

p.s. I also understood that adding your physical egress interface to an if_bridge device turns off some offloading features to the network card.

 

[cmoerz]

This is a very broad range of questions, you've got there. I'm probably not able to address everything, but I believe I've got some pointers, that may help.

One of the major things with vnet is, that you can have a completely different routing table than the host. Obviously, this can be used for security and sandboxing purposes, as you suggested.

You can use vnet to test firewall changes, for example:

Virtualize Your Network on FreeBSD with VNET - Klara Systems

Learn how to use VNET for network isolation in FreeBSD Jails. This article covers testing firewall changes and proper jail networking.

klarasystems.com

Whether you want to bridge or route really depends on your use case (i.e. do you need broadcast traffic / udp?). I.e. do you want to use a firewall to filter and validate traffic? If so, the routing option will likely be less painful, even though you can filter bridge traffic - but it appears to be a bunch harder to troubleshoot and get right.

I've always wondered whether one could use netgraph instead of epair with jails as well, but I haven't gotten around to trying it out. Here's an article introducing it in relation to bhyve:

Using Netgraph for FreeBSD’s Bhyve Networking

FreeBSD 13 adds new support for a netgraph backend for virtual network devices under bhyve. Netgraph is a modular networking framework that allows for arbitrary stacking of protocols and transports, along with filtering, tunneling, redirection, inspection, injection and more—fast and...

klarasystems.com

Not sure about the offloading functionality. That might also have to do, how you configure the device, i.e. is the IP address on the bridge or the network device? Maybe someone more knowledgeable can add some insight here.