How to implement jails --- IP addressing, routing, nat

[gpatrick]

Previously I have used OpenSolaris and network virtualization with virtual NICs and switches and zones to implement a web, dns, reverse proxy, mail setup and want to try this with FreeBSD jails.

I have read a lot of documentation but am kind of stumped on the networking part of jails. I have a cloned interface of lo1 and want to use 127.0.10 for the network. When someone tries example.com or example.org the request should hit the reverse proxy which will forward the request to the correct domain.

Do I set an IP for lo1 like 127.0.10.254/24 and the jails for www would be .2/24, rev proxy would be .3/24, mail would be .4/24, db .5/24 and then pf would use nat and rdr for only the reverse proxy? How does the www jail communicate with the db jail? if there are only a few jails such as a couple web domains and a mail and db jail, would it be simpler to have /etc/hosts entries instead of DNS? If /etc/hosts entries, would each jail need all of the other jails hosts entries?

The concept of jails is a little foreign to be because with network virtualization I can just use routers and switches to handle traffic.

Thanks for any advice and help.

 

[steinex]

This is how i solve it 99% of the time:

1) create a lo1 interface
2) add local IP addresses to that interface (like 10.0.0.1, 10.0.0.2 etc.)
3) set up your jails to use these addresses
4) add a pf-nat rule:

nat on bge0 from 10.0.0.1 to any -> $external_ip

Hint: If you want ping working inside the jails, you have to set the sysctl 'security.jail.allow_raw_sockets' to 1.

Cheers,
Frank

 

[dazealex]

Maybe you can help me -- I am trying to set up a router in a jail, which would connect to the second physical NIC connected to the modem. All I need is NAT, and some port forwarding. Is this possible to do inside a jail?

 

[SirDice]

All I need is NAT, and some port forwarding. Is this possible to do inside a jail?

With a standard jail? No.

Routing, forwarding, NAT, etc. is done on the host, not the jail.

 

[dazealex]

Other than using VirtualBox, is there a better Virtualization solution for FreeBSD? Does QEMU or KVM work?

 

[UNIXgod]

Other than using VirtualBox, is there a better Virtualization solution for FreeBSD? Does QEMU or KVM work?

http://www.freebsd.org/doc/handbook/virtualization.html

 

[gpatrick]

You can use VIMAGE. I was finally successful getting it to work, but the only problem is I was never able to successfully stop a VIMAGE Jail. Using /etc/rc.d/jail stop jailname would result in a message similar (because I didn't write it down and this is from memory)

Cannot stop jail because no jail id exists in /var/run

At start I echoed the jid to /var/run/jail_myjail.id and it did remove that using jail stop but then errored again with something different

Can't stop Jail because no host name is present

If I tried using jail -r <jid> there was a kernel panic.

Another issue was getting Jail VIMAGES to start at boot. I read different articles that had 3 different patches for /etc/rc.d/jail and I tried all three but none of them made a difference, I still couldn't start VIMAGES at boot. I finally created a script to start at boot and put

myjail_enable="YES"

in /etc/rc.conf and was able to start VIMAGE jails successfully on boot.

The ONLY change needed to the GENERIC kernel is options VIMAGE even though some reference device epair or device if_bridge they are not needed.

So far the only drawback is not being able to stop a Jail. If one needs to patch VIMAGE Jails it would require a NO in your rc.conf script and then reboot. Patch the jail and before rebooting, change it back to YES. That is the only method I have found at this time, unless someone else can suggest another method.