Almost There: CARP Issues

I spent approximately a month trying to get this working however I just don't seem to be able to. At first I tried using FreeBSD but I got to a point where preempt would just not work. I replaced all the nics but still had the same problem.

I have now installed OpenBSD4.4 and preempt seems to be workng!!!! :)

All Carp interfaces are in Master on FWA and backups on FWB. Shutting down an interface on A cause B to take over. But now I've ran into a new issue. Trying to follow the traffic using tcpdump, i can see it on carp2[ext_if] but not on carp1[dmz_if]. Traffic to the WEB or RDP server works fine until the failover occurs. Ex: i see traffice on carp1 and carp2 when FWA Master
# tcpdump -n -i carp2
tcpdump: listening on carp2, link-type EN10MB
12:05:00.541875 x.x.x.x > 192.168.1.105.3389: S 1651566250:1651566250(0) win 16384 <mss 1260,nop,nop,sackOK> (DF)
12:05:00.625594 x.x.x.x.1619 > 192.168.1.105.3389: . ack 2242405241 win 17640 (DF)

# tcpdump -n -i carp1
tcpdump: listening on carp1, link-type EN10MB
12:06:14.813558 172.16.0.189.3389 > x.x.x.x.1623: S 3718839503:3718839503(0) ack 3600434287 win 65535 <mss 1460,nop,nop,sackOK> (DF)
12:06:14.937956 172.16.0.189.3389 > x.x.x.x.1623: P 1:12(11) ack 20 win 65516 (DF)

When FWB takes over, no traffic is seem on carp1 or 2.


My PF files are pretty much identical asides from the fw_addr.

################################################################################
# Macro Defines
################################################################################
lop_int="lo0"
hrt_int="fxp0"
ext_int="rl1"
int_int="rl0"
dmz_int="rl2"

ftp_ports="{ 21,60000:60049 }"
email_ports="{ 25,110 }"
webmail_ports="{ 32000,32001 }"
ssh_fw="{ 22 }"
ssh="22"
www="{80, 443}"
vpn="1194"
rdp="3389"

vpn_server="{172.16.0.200}"
www_server="{172.16.0.80}"
rdp_server="{172.16.0.189}"

trusted_www="192.168.1.1"
trusted_vpn="192.168.1.1"

fw_addr="192.168.1.104"
dmz_nets="{172.16.0.0/24}"
internal_nets="{192.168.20.0/24}"

nat on $ext_int from $dmz_nets to any -> ($ext_int)
nat on $ext_int from $internal_nets to any -> ($ext_int)

#rdr on $ext_int proto {tcp, udp} from $trusted_vpn to any port $rdp -> $rdp_server
rdr on $ext_int proto tcp from any to any port $www -> $www_server
rdr on $ext_int proto {udp} from any to any port 1194 -> $vpn_server
#rdr on $ext_int proto tcp from any to any port 22280 -> $www_server
rdr on $ext_int proto tcp from any to port $rdp -> $rdp_server


################################################################################
# Filtering Rules
################################################################################
# Set default policy
block log all

# Log any connection attempt to the firewall
#block in log on $ext_int from any to $fw_addr

# Allow SSH Access From Trusted on External To The FW
pass in log quick on $ext_int proto tcp from any to $fw_addr port $ssh_fw keep state

# Allow VPN Traffic To VPN Server
pass in quick on $ext_int proto udp from any to $fw_addr port $vpn keep state
pass quick proto udp from any to $vpn_server port $vpn keep state

# Allow DNS To DMZ
pass quick proto {tcp,udp} from any to $dmz_nets keep state

# Allow all Loopback
pass quick on $lop_int all

# Allow pfsync Updates In/Out
pass quick on $hrt_int proto pfsync keep state

# Allow CARP Advertisements In/Out
pass quick on {$hrt_int, $ext_int, $int_int, $dmz_int} proto carp keep state

# Allow WWW Traffic To WWW Server
pass in quick on $ext_int proto tcp from any to $fw_addr port $www keep state
pass quick proto tcp from any to $www_server port $www keep state

# Allow RDP traffic To RDP Server
pass in quick on $ext_int proto tcp from any to $fw_addr port $rdp keep state
pass quick proto tcp from any to $rdp_server port $rdp keep state

# Allow SSH Traffic To WWW Server
#pass in quick on $ext_int proto tcp from any to $www_server port 22280 keep state
#pass quick proto tcp from any to $www_server port 22280 keep state


# Allow HTTP Through
pass in quick on $ext_int proto tcp from any to $www_server port $www keep state
#pass out quick on $dmz_int proto tcp from any to $www_server port $www keep state

# Allow all outgoing traffic
pass in quick on $int_int all keep state
pass out quick on $ext_int all keep state

# Allow Pings
#pass in quick on $ext_int proto icmp from any to keep state
pass out quick on $int_int proto icmp from any to any keep state

# Allow Pings to Firewall
#pass in quick on $ext_int proto icmp from any to $fw_addr keep state

# Allow Terminal Services
#pass in quick on $ext_int proto tcp from to port 3389 keep state
#pass out quick on $int_int proto tcp from to port 3389 keep state

# Allow SSL Through
#pass in quick on $ext_int proto tcp from any to port 443 keep state
#pass out quick on $int_int proto tcp from any to port 443 keep state

# Allow FTP Through
#pass in quick on $ext_int proto tcp from any to port $ftp_ports keep state
#pass out quick on $int_int proto tcp from any to port $ftp_ports keep state

# Allow Email Through
#pass in quick on $ext_int proto tcp from any to port $email_ports keep state
#pass out quick on $int_int proto tcp from any to port $email_ports keep state

# Allow Webmail Through
#pass in quick on $ext_int proto tcp from any to port $webmail_ports keep state
#pass out quick on $int_int proto tcp from any to port $webmail_ports keep state

# Allow DNS Through
#pass in quick on $ext_int proto { tcp, udp } from any to port 53 keep state
pass out quick on $int_int proto { tcp, udp } from any to port 53 keep state

Any help would be much appreciated, I need this up and running for a school project which is comming up next week.
 
Here's my configs:

FWA
inet 10.1.0.1 255.255.255.0 10.1.0.255 vhid 1 pass 001vCarpAdvert
inet 172.16.0.1 255.255.255.0 172.16.0.255 vhid 2 pass 001vCarpAdvert
inet 192.168.1.105 255.255.255.0 192.168.1.255 vhid 3 pass 001vCarpAdvert
inet 192.168.20.105 255.255.255.0 192.168.20.255 vhid 4 pass 001vCarpAdvert
inet 10.1.0.103 255.255.255.0 NONE
up syncif fxp0
inet 192.168.20.1 255.255.255.0 NONE
inet 192.168.1.103 255.255.255.0 NONE
inet 172.16.0.103 255.255.255.0 NONE

FWB
inet 10.1.0.1 255.255.255.0 10.1.0.255 vhid 1 pass 001vCarpAdvert advskew 100
inet 172.16.0.1 255.255.255.0 172.16.0.255 vhid 2 pass 001vCarpAdvert advskew 100
inet 192.168.1.105 255.255.255.0 192.168.1.255 vhid 3 pass 001vCarpAdvert advskew 100
inet 192.168.20.105 255.255.255.0 192.168.20.255 vhid 4 pass 001vCarpAdvert advskew 100
inet 10.1.0.104 255.255.255.0 NONE
up syncif fxp0
inet 192.168.20.104 255.255.255.0 NONE
inet 192.168.1.104 255.255.255.0 NONE
inet 172.16.0.104 255.255.255.0 NONE
 
ok, rebooting the servers has brought traffic to carp2[ext_if] after failing FWA.

FWB# tcpdump -n -i carp2
tcpdump: listening on carp2, link-type EN10MB
13:56:30.036500 173.32.139.31.2081 > 192.168.1.105.3389: S 1381720291:1381720291(0) win 16384 <mss 1260,nop,nop,sackOK> (DF)
13:56:32.929226 173.32.139.31.2081 > 192.168.1.105.3389: S 1381720291:1381720291(0) win 16384 <mss 1260,nop,nop,sackOK> (DF)
13:56:38.958281 173.32.139.31.2081 > 192.168.1.105.3389: S 1381720291:1381720291(0) win 16384 <mss 1260,nop,nop,sackOK> (DF)

# tcpdump -n -i carp1
tcpdump: listening on carp1, link-type EN10MB
*nothing*

pfsync0: flags=41<UP,RUNNING> mtu 1460
pfsync: syncdev: fxp0 maxupd: 128
groups: carp pfsync
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33204
groups: pflog
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:01
carp: MASTER carpdev fxp0 vhid 1 advbase 1 advskew 100
groups: carp
inet 10.1.0.1 netmask 0xffffff00 broadcast 10.1.0.255
inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x7
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:02
carp: MASTER carpdev rl2 vhid 2 advbase 1 advskew 100
groups: carp
inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255
inet6 fe80::200:5eff:fe00:102%carp1 prefixlen 64 scopeid 0x8
carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:03
carp: MASTER carpdev rl1 vhid 3 advbase 1 advskew 100
groups: carp
inet 192.168.1.105 netmask 0xffffff00 broadcast 192.168.1.255
inet6 fe80::200:5eff:fe00:103%carp2 prefixlen 64 scopeid 0x9
carp3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:04
carp: MASTER carpdev rl0 vhid 4 advbase 1 advskew 100
groups: carp
inet 192.168.20.105 netmask 0xffffff00 broadcast 192.168.20.255
inet6 fe80::200:5eff:fe00:104%carp3 prefixlen 64 scopeid 0xa
 
Back
Top