CARP and routing with two interfaces

[ramirez]

Hi guys!

I have question about routing configuration with two network interfaces and CARP enabled.

I have FreeBSD 10.1-RELEASE amd64 with two network interfaces: one for public services with CARP enabled and second for management.

Each interface has one IP. Both from same network.

my config:

defaultrouter="1.1.1.1"
ifconfig_em0="inet 1.1.1.2/28 vhid 3 pass testpass"
ifconfig_em1="inet 1.1.1.3/28"

Works fine, but default route always stick to interface em1:

netstat -rn
Routing tables

Internet:
Destination  Gateway  Flags  Netif Expire
default  1.1.1.1  UGS  em1
1.1.1.1/28  link#2  U  em1
1.1.1.2  link#1  UHS  lo0
1.1.1.3  link#2  UHS  lo0
127.0.0.1  link#3  UH  lo0

In this case, all return traffic goes trough "management" interface em1.

If I manually configure defaultroute trough em0, all works as needed: all traffic goes via em0.
But, i think, if em0 falls (or preempted by other host), then i lose connections to management interface em1 too.

Looks like I'm doing something wrong

What is right configuration in this case?

Thanks!

 

[SirDice]

Interfaces should each be in it's own subnet. Routing isn't going to work when both interfaces are in the same subnet.

 

[ramirez]

Thanks!

Policy Based Routing (iI use PF) or setfib can be useful in my current configuration?

Looks like iI have two solutions:
1. use only one network interface and two IP from single subnet;
2. use two interfaces with two IP different subnets (and how my routing configuration will look like in this case?).

 

[junovitch@]

Something along the lines of this might be enough.
/boot/loader.conf

net.fibs=2

/etc/rc.conf

defaultrouter="1.1.1.1"
ifconfig_em0="inet 1.1.1.2/28 vhid 3 pass testpass"
ifconfig_em1="inet 1.1.1.3/28 fib 1"
static_routes="mgmtdefault"
route_mgmtdefault="default 1.1.1.1 -fib 1"
sshd_fib=1  #Just guessing on this one, but makes sense if the FIB is for management