Solved FreeBSD as a MITM router

[plexinvise]

Hi there, I am stuck with a problem I cannot resolve myself, unfortunately.

I have a RPI3 device with FreeBSD on it, which I want to act as a router with firewall. Mu initial intent to use it as a sort of MITM router that I can control and adapt firewall needs for the purposes of testing network connectivity on target device. Initial setup was a transparent bridge for two interfaces ue0 (internet connection) and ue1 (client connection). Now I need to change it to be able to use dnsmasq for custom dns resolution. The way I set it up right now:

ifconfig

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
        ether b8:27:eb:ec:b9:ed
        inet 192.168.50.70 netmask 0xffffff00 broadcast 192.168.50.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ue1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether a0:ce:c8:5e:c7:1d
        inet 192.168.31.1 netmask 0xffffff00 broadcast 192.168.31.255
        media: Ethernet autoselect (1000baseT <full-duplex,master>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ue0 facing the internet
ue1 facing the client

dhcpd.conf

# DHCPD config
option domain-name "void.com";
option domain-name-servers 192.168.31.1;
default-lease-time 600;
max-lease-time 7200;
authoritative;
subnet 192.168.31.0 netmask 255.255.255.0 {
  range 192.168.31.10 192.168.31.254;
  option broadcast-address 192.168.31.255;
  option routers 192.168.31.1;
}

I am also running dnsmasq as my dns server

dnsmasq.conf

no-resolv
log-queries
interface=ue1
no-dhcp-interface=ue1
server=1.1.1.1
server=1.0.0.1
address=/example.com/1.2.3.4

and here is /etc/rc.conf

ifconfig_ue0="DHCP"
growfs_enable="YES"
powerd_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
#syslogd_enable="NO"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Enable Firewall
firewall_enable="YES"
firewall_type="OPEN"
gateway_enable="YES"

# Setup DHCPD on ue1
dhcpd_enable="YES"
ifconfig_ue1="inet 192.168.31.1 netmask 255.255.255.0"
dhcpd_ifaces="ue1"

# Enable dummynet
dummynet_enable="YES"
dnsmasq_enable="YES"

So the problem I am facing is that I get the IP on client, I get dns resolution on client too. But I cannot even ping any external (internet) addresses.

PING google.com (142.251.40.238): 56 data bytes
Request timeout for icmp_seq 0

However I can ping gateway 192.168.31.1 and I can ping the external interface on the freebsd machine 192.168.50.70. And as I mentioned DNS is also working fine and resolves addresses as expected.

Any help will be appreciated.

Thank you

 

[SirDice]

Initial setup was a transparent bridge

A transparent bridge is a tautology, a bridge is by definition transparent. A bridge is a layer 2 connection.

But I cannot even ping any external (internet) addresses.

There's a router on your network that actually connects the network of ue0 to the internet. It needs to know where it can find the 192.168.31.0/24 network or else it won't be able to send the return packets. So you'll need to add a static route there that tells it to find 192.168.31.0/24 behind whatever address was assigned to ue0. You don't want to use DHCP here as the address could change, you want it to have a static address.

 

[plexinvise]

A transparent bridge is a tautology, a bridge is by definition transparent. A bridge is a layer 2 connection

Thank you, now I know. I am still very much green with networking.

There's a router on your network that actually connects the network of ue0 to the internet. It needs to know where it can find the 192.168.31.0/24 network or else it won't be able to send the return packets. So you'll need to add a static route there that tells it to find 192.168.31.0/24 behind whatever address was assigned to ue0. You don't want to use DHCP here as the address could change, you want it to have a static address.

Ok, I see your point. What if I have no control over upstream router? Want to make it as independent as possible from the network it is connected to.

I was thinking that I should have nat configured but I have no idea what would be a sufficient configuration in this case. Also I am using ipfw right now for firewall.

That said maybe you can direct me towards a nandbook chapter that I need to understand. I’ve read majority if it but some of it may went over my head.

Thank you

 

[SirDice]

What if I have no control over upstream router?

Then your only option is to NAT the traffic.

I was thinking that I should have nat configured but I have no idea what would be a sufficient configuration in this case.

Yes, NAT would be the solution. Don't know IPFW that well, but for PF it's as simple as nat on ue0 from 192.168.31.0/24 to any -> (ue0)

That said maybe you can direct me towards a nandbook chapter that I need to understand.

You won't find this in the handbook. This is "routing 101" and basic TCP/IP knowledge. So if you want to learn more get a good book that covers TCP/IP.

 

[plexinvise]

Then your only option is to NAT the traffic.


Yes, NAT would be the solution. Don't know IPFW that well, but for PF it's as simple as nat on ue0 from 192.168.31.0/24 to any -> (ue0)

Thank you. I will go dig through documentation. Will post here when I have a solution.

 

[plexinvise]

Thank SirDice again for pointing it out for me. As soon as I knew what to search for I quickly found a solution.
I needed to add NAT for the connection and the following rules worked.

ipfw nat 1 config if ue0
ipfw add 00001 nat 1 ip from any to me in via ue0
ipfw add 00002 nat 1 ip from 192.168.31.0/24 to any out via ue0

The rest of the firewall configured as OPEN so nothing special there.

Thanks again

 

[smithi]

Then your only option is to NAT the traffic.

Yes, NAT would be the solution. Don't know IPFW that well, but for PF it's as simple as nat on ue0 from 192.168.31.0/24 to any -> (ue0)

plexinvise already has mostly what's needed, namely an 'open' ipfw firewall. This passes everything, but also does NAT before anything else if firewall_nat_enable="YES" in rc.conf , with firewall_nat_flags set appropriately - see /etc/rc.firewall NAT section for details.

You won't find this in the handbook. This is "routing 101" and basic TCP/IP knowledge. So if you want to learn more get a good book that covers TCP/IP.

Sure, but there's a good article (as .html or .pdf) in
/usr/local/share/doc/freebsd/en/articles/filtering-bridges/.

I used (an earlier version of) this c.2004 for an essentially similar purpose, between a ISDN router we had no control over providing DHCP addrs and a dozen client boxes, albeit plus traffic accounting and some blocking.

Of course that setup needs mods to allow RFC1918 addresses either side of the bridge, and setting the appropriate bridge sysctls.

 

[plexinvise]

plexinvise already has mostly what's needed, namely an 'open' ipfw firewall. This passes everything, but also does NAT before anything else if firewall_nat_enable="YES" in rc.conf , with firewall_nat_flags set appropriately - see /etc/rc.firewall NAT section for details.



Sure, but there's a good article (as .html or .pdf) in
/usr/local/share/doc/freebsd/en/articles/filtering-bridges/.

I used (an earlier version of) this c.2004 for an essentially similar purpose, between a ISDN router we had no control over providing DHCP addrs and a dozen client boxes, albeit plus traffic accounting and some blocking.

Of course that setup needs mods to allow RFC1918 addresses either side of the bridge, and setting the appropriate bridge sysctls.

Thank you. I’d complain that I went through a lot of googling to make it work when I could have just enabled ipfw_nat and this problem would be solved, but hey, I’ve learned something so that counts.

And thank you for pointing out resources to read, I will get on it

 

[smithi]

there's a good article (as .html or .pdf) in
/usr/local/share/doc/freebsd/en/articles/filtering-bridges/.

OT, but could someone running 13.2 or 14 please confirm that .html versions of a) articles and b) handbook still exist?

ISTR something suggesting .pdf- only but I may misremember, and discussion re handbook is still WIP.

 

[smithi]

Thank you. I’d complain that I went through a lot of googling to make it work when I could have just enabled ipfw_nat and this problem would be solved, but hey, I’ve learned something so that counts.

And thank you for pointing out resources to read, I will get on it

Well layer2 filtering is different enough that just enabling NAT may not be everything needed.

If in doubt, add temporary 'count log' rules, perhaps before and after NAT rules, so you can fully trace operation with various scenarios in /var/log/security

Refer to the ipfw(8) section "PACKET FLOW" if you need to distinguish layer2 from layer3 (IP) packets. Enjoy!

 

[SirDice]

OT, but could someone running 13.2 or 14 please confirm that .html versions of a) articles and b) handbook still exist?

misc/freebsd-doc-en for the English version.

 

[plexinvise]

Well layer2 filtering is different enough that just enabling NAT may not be everything needed.

If in doubt, add temporary 'count log' rules, perhaps before and after NAT rules, so you can fully trace operation with various scenarios in /var/log/security

Refer to the ipfw(8) section "PACKET FLOW" if you need to distinguish layer2 from layer3 (IP) packets. Enjoy!

oh man, thank you! That is exactly what I needed to troubleshoot! For some reason I haven't even thought to count packets.
On the other note, how do I mark the thread as solved?
Found it. Thank you everybody for the help!