IPFW How to avoid CARP's IP interference with IPFW NAT rules?

supportsobaka

Member


Messages: 34

To pass traffic from a remote host 55.55.55.55 to a service behind the NAT on the box with IP 77.77.77.77 I have a standard set of rules:

Code:
00812 nat 82 tcp from 55.55.55.55 to 77.77.77.77 48888 in via igb0
00822 allow tcp from 55.55.55.55 to 10.1.1.8 48888 in via igb0
00832 nat 82 tcp from 10.1.1.8 48888 to 55.55.55.55 out via igb0
00842 allow tcp from 77.77.77.77 48888 to 55.55.55.55 out via igb0

where CARP is configured for 55.55.55.55 on that box, therefore 55.55.55.55 is presented on igb0:

Code:
igb0:   
        inet 77.77.77.77 netmask 0xffffffff broadcast 77.77.77.77
        inet 55.55.55.55 netmask 0xffffffff broadcast 55.55.55.55 vhid 1
        carp: BACKUP vhid 1 advbase 1 advskew 100

and rules will not work.

Traffic hits only rules 00812 and 00822 when 55.55.55.55 is there on the NIC. No other counters incremented, but only for these two rules. Neither there are rules that could match the packet which leaves ipfw on the rule 00822. It's just presence of 55.55.55.55 for failover purposes creates the problem. Once removed, obviously, rules work.

I couldn't find how to improve rules for such conditions based on vhid reference, for example. The man seems doesn't mention vhid. Looks like lack of options for ipfw in case of using CARP.

Questions:

Is there any "magic" trick to say IPFW ignore CARP's IP presented on the NIC? Why does this problem exist at all; where packet after rules 00822 "stuck"/"lost"?

FreeBSD 11.3
ipfw nat 82 config ip 55.55.55.55 log same_ports unreg_only reset redirect_port tcp 10.1.1.8:48888 48888
net.inet.ip.fw.one_pass=0, though this irrelevant to the question.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,294
Messages: 38,811

To pass traffic from a remote host 55.55.55.55 to a service behind the NAT on the box with IP 77.77.77.77
Both IP addresses are assigned to the host according to the igb0 output. So 55.55.55.55 isn't "remote", this is actually your CARP VIP address. 77.77.77.77 is the actual IP address of that host. So, this host now as two IP addresses, one for the host itself and one CARP VIP address. And in this case, the VIP address isn't active (it's set as BACKUP).
 
OP
S

supportsobaka

Member


Messages: 34

Both IP addresses are assigned to the host according to the igb0 output. So 55.55.55.55 isn't "remote", this is actually your CARP VIP address. 77.77.77.77 is the actual IP address of that host. So, this host now as two IP addresses, one for the host itself and one CARP VIP address.

It is obvious.
Solution?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,294
Messages: 38,811

Your explanation of what you want doesn't match with what you're showing. So, what exactly are you trying to do?
 
OP
S

supportsobaka

Member


Messages: 34

Your explanation of what you want doesn't match with what you're showing
How is that?

I want pass traffic from remote host 55.55.55.55 to the service running on 10.1.1.8 on the box 77.77.77.77. IP 55.55.55.55 naturally is also presented on the interface of that box since the essence of the fail-over using CARP suppose it to be there.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,294
Messages: 38,811

I want pass traffic from remote host 55.55.55.55 to the service running on 10.1.1.8 on the box 77.77.77.77. IP 55.55.55.55 naturally is also presented on the interface of that box since the essence of the fail-over using CARP suppose it to be there.
Contradictio in terminis. 55.55.55.55 is NOT a remote host because it's a local address of the 77.77.77.77 host (it's the CARP VIP address). Assume for a second there's no CARP and this host has two IP addresses, 55.55.55.55 and 77.77.77.77. Then how can 55.55.55.55 be "remote"?

Are you talking about the other node of the CARP? Then you shouldn't be pushing traffic from the active node to the passive node. That's going to cause problems. Both nodes need to be connected to the same things and only one of them should be active. So the 10.1.1.8 has to be reachable from both nodes too.
 
OP
S

supportsobaka

Member


Messages: 34

It's NOT a remote host for that machine ONLY because it's PART OF CARP and it suppose to be there on the interface:

carp: BACKUP vhid 1 advbase 1 advskew 100

Otherwise, the host 55.55.55.55 is currently MASTER on another box, from which the traffic should be passed.

That's how CARP is designed.

So, ipfw is lack of features to manage this?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,294
Messages: 38,811

Otherwise, the host 55.55.55.55 is currently MASTER on another box, from which the traffic should be passed.
No, it should only be passed on the active (i.e. MASTER) node. Don't push traffic from the active node to the passive. That's going to cause weird asymmetrical routing, which causes all sorts of issues.

The whole point of CARP is to provide active/passive fail-over. The passive node (BACKUP) can be completely turned off and things will continue to work. Trying to push traffic from the active node to the passive node will break this (the BACKUP could never be turned off).
 
OP
S

supportsobaka

Member


Messages: 34

Don't push traffic from the active node to the passive. That's going to cause weird asymmetrical routing, which causes all sorts of issues.
You know my setup precisely?.. I know exactly what I'm doing with CARP and I don't break any standards or general idea about CARP with what I'm doing with CARP. I am not asking about CARP...

It's not the question about CARP, but about IPFW and how IPFW can be improved to allow ignoring presence of CARP's IP on the interface, i.e. don't consider it local. The best options would be to "detect" it via vhid reference, but nothing about it in the user manual.
 

VladiBG

Daemon

Reaction score: 554
Messages: 1,201

It's not about CARP, it's about how you do NAT. Read the manual here how to redirect ports.
 
OP
S

supportsobaka

Member


Messages: 34

Which is the crux of the problem. I can't properly answer based on the information you have provided. So I have to make assumptions.

You just need not to think about my CARP configuration at all, but about the fact that some traffic from box1 with IP 55.55.55.55 (origin/source will be 55.55.55.55) should be passed to box2 with IP 77.77.77.77 (that also has CARP's IP 55.55.55.55 on the same NIC), and this trafic belongs to the service running on 10.1.1.8 on that box2.

You think all CARP configurations are the same? Datacenter will NOT route box2 to the world by IP 55.55.55.55 in my case until I switch this routing in the datacenter (by means of another script running on the independent node). The 55.55.55.55 from the box2 doesn't exist for the world by default. Please, don't tell me that I don't need CARP on the box2 in that case... I know exactly what I'm doing with CARP. It's not crux of the problem. I never asked how to configure CARP or similar. This post to IPFW section, not CARP. And yes, I know that if I would remove that IP from the interface then rules work - I said this in my initial post.

The question is as simple as: how to tell IPFW that CARP's IP is not really our local (e.g. based on some identification, e.g. "vhid").
 

VladiBG

Daemon

Reaction score: 554
Messages: 1,201

instead of IP address use nic alias for interfaces that have dynamical ip address.

if nic Use ip address of NIC for aliasing, dynamically changing it if
NIC's ip address changes.
 
Top