Creating virtual network devices for jails

S

SageRaven

I'm testing out a FreeBSD VPS with FBSD 8.1RC2. I've got 1 "real" ethernet interface, which has my public IP. The plan is to jail everything on non-routable IPs and use PF to redirect traffic to the internal jails.

I installed 2 DNS server jails: one for internal caching and one to be my domain's primary name server. I ran into the problem of jails on the same interface (my only one) not being able to bind to the same port.

I considered binding one jail to the ethernet interface and the other to the loopback, but just seemed wrong somehow. So, I figured there must be a way to create some kind of virtual interface, one for each jail, so that I wouldn't need to even worry about each jail stepping on each other.

Google wasn't much help (maybe my google fu was off that day), so after many an "apropos" and much poking through /etc/defaults/rc.conf, I decided to use "cloned_interfaces" to create bridge0 and bridge1 without actually bridging them to anything. The jails bind to them just fine, and work as you'd expect.

But something tells me that there must be a more "proper" way to do this. Maybe a driver specifically meant for this? Or maybe with the new virtualized network stack for jails? In any case, I'd appreciate if anyone with more knowledge of FBSD networking arcana would tell me if my chosen method is acceptable or if there is a more proper method of accomplishing what I want to do.
 
I don't think the network virtualisation project is that far yet. The norm is to assign IP aliases to your ethernet device and assign those IPs to jails.
 
Which network device for jails?

I face a similar situation. My freeBSD machine also has one real ethernet interface, which has my public IPs assigned to it and I want to put single network services into jails with private IPs. PF is designated to redirect traffic between the internal jails and the internet.

My question is now to which network device shall I attach the private IPs and therewith the jails? Intensive internet search brought multiple suggestions (assign them to the ethernet card's interface, to lo0, to a cloned loopback device (e.g. lo1), to a bridged network that bridges to "nowhere", ...) but not a single explanation why the suggested way is the right one.

That's why I ask here now: What is the default way to solve this problem and why is that the best or at least the intended solution?
 
The 'normal' way to use jails is to create aliases on the network interface and attaching the jails to that.

But if you want to use RFC1918 addresses and your nic has a public ip address the best way would probably be to clone the loopback interface and assign the private addresses to that. Then you can use PF (or one of the others) to NAT/redirect traffic to the jails.

On *nix there usually isn't a 'default' way. You can use any kind of solution if it works for you. Most of the time when you ask 10 people how to solve it you'll get 10 different solutions ;)
 
These may shed some light:

http://wiki.freebsd.org/Image/VNETSamples

http://bsdbased.com/2009/12/06/freebsd-8-vimage-epair-howto

http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet

I've experimented with a variety of setups, from network simulation to hosting services in vnet jails, and I directed packet flow with IPFW and/or multiple routing tables.

As the network virtualization project is still labeled "experimental", documentation is relatively sparse -- but the examples available and the source code is enlightening :)
 
On one server I created lo1 interface and added local a subnet and ip-addresses to the lo1 device. Used natd to reverse-nat specific ports to relevant jails. Works like a charm for me.

Code: 
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet 192.168.99.1 netmask 0xffffff00
        inet 192.168.99.2 netmask 0xffffff00
Snippet from natd.conf (webserver is running on the .2 host)
Code: 
redirect_port tcp 192.168.99.2:80 80
and for the jail
Code: 
export jail_http_ip="192.168.99.2"
 
I didn't consider making an additional loopback interface. That might be easier. I thought I read somewhere that binding a lot of stuff to loopbacks can cause weird problems due to 127./8 addresses being treated rather uniquely. You've never had any odd issues with this setup?
 
You must log in or register to reply here.
Back
Top