Solved Building a networking lab with FreeBSD 13 in VirtualBox; virtual LAN unable to ping 8.8.8.8

[del]

I'm trying to build a multi-router setup in VirtualBox using FreeBSD machines as the routers. I have two VMs, router-1 and router-2, which are on the same subnet and are able to ping one another.

router-1 has a bridge interface with the host machine and is able to ping the wider internet. It gets it's IP dynamically from my home network's gateway.

router-2 has a LAN interface configured to connect to router-1.

All of the remaining interfaces for both VM's are set up so I can SSH into the machines with host-only adapters. Each VM is configured to use a separate host-only interface.

I would like to avoid using DHCP. I know that might seem silly, but I'm using this as a learning exercise and want to do as much manually as possible, at least at first.

Here's the /etc/rc.conf for router-1:

hostname="router-1"

growfs_enable="YES"

# WAN --------- configuration for bridged adapter
ifconfig_em0="dhcp"

# SSH --------- configuration for host-only adapter in virtualbox (vboxnet0); needed for ssh
sshd_enable="YES"
ifconfig_em1="inet 192.168.99.1 netmask 255.255.255.0"

# LAN_0 ------- configuration for virtualbox internal network interface: routerNet1
ifconfig_em2="inet 10.0.0.1 netmask 255.255.255.0"

And the /etc/resolv.conf for router-1:

# Generated by resolvconf
search attlocal.net
nameserver 192.168.1.254

And the netstat -r for router-1:

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.0.0/24        link#3             U           em2
router-1           link#3             UHS         lo0
localhost          link#4             UHS         lo0
192.168.1.0/24     link#1             U           em0
192.168.1.206      link#1             UHS         lo0
192.168.99.0/24    link#2             U           em1
192.168.99.1       link#2             UHS         lo0

Here's the /etc/rc.conf for router-2:

hostname="router-2"

growfs_enable="YES"

# Gateway -------------- enable internetworking(?)
defaultrouter="10.0.0.1"
gateway_enable="YES"

# SSH ---------------- configuration of host-only adapter in vitualbox (vboxnet1); needed for ssh
sshd_enable="YES"
ifconfig_em0="inet 192.168.100.1 netmask 255.255.255.0"

# LAN_0 ------------- configuration for virtualbox internal network interface: routernet1
ifconfig_em1="inet 10.0.0.2 netmask 255.255.255.0"

And the /etc/resolv.conf for router-2:

nameserver 10.0.0.1

And the netstat -r for router-2:

Internet:
Destination        Gateway            Flags     Netif Expire
default            router-1.my.domain UGS         em1
10.0.0.0/24        link#2             U           em1
10.0.0.2           link#2             UHS         lo0
localhost          link#3             UHS         lo0
192.168.100.0/24   link#1             U           em0
192.168.100.1      link#1             UHS         lo0

I'm sure that I'm missing something fundamental, but I'm not sure how to frame further inquiry intelligibly. Any advice is much appreciated.

 

[del]

Well it turns out I was missing a bunch of fundamental stuff.

I needed to set gateway_enable="YES" on router-1, then I also needed to configure /etc/pf.conf to manage packet forwarding.

It works now, but I'm sure it needs a lot of cleanup.

 

[SirDice]

then I also needed to configure /etc/pf.conf to manage packet forwarding.

Not unless you used NAT, which you shouldn't need to use in this case as you can handle this with plain old routing. You do need to allow the traffic to pass of course.

 

[del]

Not unless you used NAT, which you shouldn't need to use in this case as you can handle this with plain old routing. You do need to allow the traffic to pass of course.

Interesting. I must be inadvertently using NAT, since I'm unable to route through to the internet from router-2 when gateway_enable="YES" is enabled on router-1 without having pf configured and enabled as well.

I suppose I may be inadvertently allowing traffic to pass through when using pf, but what would I put in /etc/rc.conf to allow traffic without using pf?

So, here's a dumb question: Since router-2 only has a single interface on the 10.0.0.0/24 network, and router-1's WAN interface is on the 192.168.1.0/24 network... is that why I had to enable pf? I mean, that does sound (to an absolute novice) like I'm translating between two networks.

 

[SirDice]

You need PF on router-1 because it connects to the internet, so you definitely need to use NAT on em0 of router-1. Unless em0 is connected to a modem/router that already takes care of the NAT to internet. In that case you don't need NAT on router-1. But you do need to add a couple of static routes on your modem/router in that case. Your modem/router needs to know that 192.168.99.0/24, 192.168.100.0/24 and 10.0.0.0/24 can be found by sending the traffic to router-1.

Traffic between router-1 and router-2 is based on the routing tables. PF has nothing to do with this. On router-1 you're going to need to add a static route for 192.168.100.0/24 so it knows it needs to send that traffic to router-2. Router-2 knows where it is as it's a directly connected network there. All traffic going out of router-2 already gets sent to router-1 due to its default gateway. So you don't need static routes there.

Both router-1 and router-2 need gateway_enable in order to get them to route traffic at all.