fibs in 10.0R jails

T

tajudd

Hello,

I've got a multihomed PC that is currently setup with the 10.0R (redesigned) jail(8)s, and I'm having issues getting the setfib() altered routing table to effect in the jail. The default fib (0) is applied when I set exec.fib globally or in an instance of a jail. I have to unhide /dev/mem in order to view the routing tables within the jail, and asking to show the jail's default routing table shows fib 0 when the exec.fib=1 should be setting fib 1 for the jail instance. I need a little sanity check and another pair of eyes to see if I'm doing something wrong, or if the new jail system is possibly ignoring the fib settings.


This is 10.0p7 i386 with 1x em(4) and 2x xl(4) cards.
Please - ask questions and I'll provide feedback. I will revert outside the jail if the jail is unable to provide this functionality
 
tajudd said:
I have to unhide /dev/mem in order to view the routing tables within the jail, and asking to show the jail's default routing table shows fib 0 when the exec.fib=1 should be setting fib 1 for the jail instance. I need a little sanity check and another pair of eyes to see if I'm doing something wrong, or if the new jail system is possibly ignoring the fib settings.
Click to expand...

I actually see this /dev/mem warning message in normal jails that never had their FIB changed. I've used FIB on my jails for a year and never noticed. I think that behavior makes sense as jails prevent you from modifying a routing table. It doesn't make sense to need to look at something you can't change. Here is how I setup FIBs from my notes.

1. Enable it. A reboot is required to make this take effect so setting this a bit higher than needed makes sense.
echo 'net.fibs=4' >> /boot/loader.conf

2. If you need to prevent the directly connected interfaces from getting added by default to each FIB, in the use case of keeping a jail bound to one NIC from using other other NIC on a dual-homed host, add this line as well. I have found this works perfectly for IPv4 but IPv6 addresses still get added to all FIBs by default.
echo 'net.add_addr_allfibs=0' >> /boot/loader.conf

3. Update rc.conf with the appropriate networking bits. This will stick a certain aliases's IP to a FIB and add some routes to it to talk to the local subnet and default gateway. Expand as need of course.
Code: 
ifconfig_em0_alias0="inet 192.168.102.11/32 fib 2"
static_routes="fibnetwork fibdefault"
route_fibnetwork="-net 192.168.102.0/24 -interface em0 -fib 2"
route_fibdefault="default 192.168.102.1 -fib 2"

4. Finally, update the application or jail accordingly. I'm using sysutils/ezjail so I do it this way.
ezjail-admin config -f 2 jail.example.com

You still won't be able to see the routing table from the jail and I'm betting that if you allow access to /dev/mem it would still show FIB 0. However, tcpdump clearly shows packets leaving out the interface and FIB they belong to.
 
I abandoned the fib route, and set a static route to the NIC subnet to the broader 10.0.0.0/8 range. While not exactly what I wanted, it is serving my needs. I appreciate the feedback and it did help in the sanity check I often find myself without. Thank you, --Tim
 
You must log in or register to reply here.
Back
Top