Accessing CARP interface from localhost

[sirual]

So I have a simple CARP setup (bare-metal) which was working fine until I upgraded to 13.

Basically, external clients can reach (and ping) CARP ip, no problem. However, if I login into MASTER CARP server, I can't reach services running on CARP ip address. I can't even ping CARP ip (ping: sendto: Host is down) :/. I'm missing something obvious here, but can't figure out what.. Any ideas ?
(Firewall is disabled)

 

[mickey]

Is your CARP server by chance running on an IPv4 alias address? There have been changes to the way routes for alias IP addresses are created in FreeBSD 13, up to FreeBSD 12.2 these routes would point to the loopback interface, now they dont, check netstat -rn for the particular address and note what it says in the netif column.

 

[sirual]

I think you nailed it, mickey. That's how CARP address is defined:

ifconfig_igb0_alias0="inet vhid 1 advskew 100 pass xxx alias 10.1.1.42/32"

And that's how routing table looks like:

Destination        Gateway            Flags     Netif Expire
default            10.1.1.1           UGS        igb0
10.1.1.0/24        link#1             U          igb0
10.1.1.41          link#1             UHS         lo0
10.1.1.42          link#1             UH         igb0
127.0.0.1          link#5             UH          lo0

So what would be recommended/best practice way to bring back 12.2 behavior ?

 

[mickey]

And that's how routing table looks like:

Destination        Gateway            Flags     Netif Expire
10.1.1.42          link#1             UH         igb0

That seems to be the culprit. Prior to FreeBSD 13 this route used to point to the loopback interface lo0, now it instead points to the physical interface. I'm seeing similar behaviour with an apache web server running on an alias IP, a setup I haven't touched in many years and up to FreeBSD 12.2 it was working. Now with FreeBSD 13 the server can be accessed externally, but not from the host itself. I found that connection attempts from the host itself are now blocked by my pf(4) firewall, which is intended behaviour as such traffic is not supposed to go out on igb0 in the first place.

So what would be recommended/best practice way to bring back 12.2 behavior ?

I'm still looking for a definitive solution.
Manually deleting the route and adding one with lo0 as the interface seems to restore the old behaviour:

# route del 10.1.1.42
# route add 10.1.1.42 -interface lo0

But that is hardly a long-term solution. Creating the IP alias with the 'real' netmask instead of /32 seems to create a route like in previous releases, but all documentation suggests that /32 is correct for alias IPs:

ifconfig igb0 inet 10.1.1.42 -alias
ifconfig igb0 inet 10.1.1.42/24 alias

So far I could only find a few clues regarding this issue.
The FreeBSD 13.0 release notes mention: Duplicate routes installation issue for /32 or /128 interface aliases has been fixed. 81728a538d24
Then there are also D30811 and D28246 as well as some related bug reports 256681 and 258949.
Some of the comments suggest that it is working as intended, which for us it clearly is not.

 

[sirual]

I guess I'll stick with your first suggestion for now, unless it will be "fixed" in a dot release or smth, but, as you mentioned, I'm not holding my breath . Thanks a lot mickey!