Routing wifi traffic through Linux Bhyve guest

[tOsYZYny]

My host is using the LAN interface in the network switch create via vm-bhyve and for the Linux guest. Inside the Linux guest, I am using pci passthru to work directly with the wifi adapter. Both the Ethernet and wifi adapters in the Linux guest are in a bridge which has a static IP belonging to the same subnet as the LAN interface. The Linux guest can get out to the Internet just fine. Neither the Ethernet nor the wifi adapters has an IP directly assigned to it.

The Linux guest has hostapd running and I am able to connect to it, but I cannot get a DHCP lease from the DHCP server listening on the LAN interface. I do see DHCP traffic on the wifi interface, but it isn't getting to the DHCP server listening on the LAN interface.

1. with a bridge, shouldn't the request get through? If not, would a DHCP relay work here? I tried a relay and it discarded all of the traffic because the wifi interface didn't have an IP address assigned to it.

2. excluding DHCP, and using a static IP on a client, I was still unable to get out to the Internet. I had setup a default gateway on the bridge. Since the client and bridge are on the same subnet as the LAN interface on the host which is the gateway, do I need to enable NAT?


Since the VM is on the same subnet, it doesn't make sense to me that I would need NAT. I was trying to keep this high-level to see if my understand was correct, I can provide more details as to IP addresses if that helps to clarify what I'm trying to do.

 

[tOsYZYny]

Hopefully this helps someone else.

1. I switched to Alpine Linux (virt) as it is much smaller and easier to automate via expect scripts. My wifi router setup in Alpine consumes 120MB for root! Ubuntu wanted more memory and out of the box took nearly 2GB of disk. Ubuntu works, but I want to automate the installation so the process is repeatable.
2. I switched to the linux-lts kernel as it provides modules / drivers for more than the virt package (install linux-firmware-none first, then linux-lts and any firmware / drivers you need, then uninstall linux-virt). Poweroff the VM and be sure to switch from the virt kernel and init to the lts. If you don't do it in this order, it won't work.
3. Power on the VM, configure the default interface, eth0, in a bridge with wlan0
4. Configure hostapd with wlan0 and the bridge

Replace wlan0 with the actual name you're using.

That is it, I didn't need to make any changes to forwarding, use a DHCP forwarder, etc.

I'm using an Asus PCEAC88 and trying to get the full potential out of the card. Right now, I'm getting a whopping 72Mbps ! It is rated for 3100Mbps.

 

[tOsYZYny]

For some reason I have yet to figure out, routing is no longer working. I'm not sure precisely when the problem started occurring. Disabling the firewall altogether doesn't resolve the problem, so I suspect it has something to do with the bridge and tap0 / switch.

Randomly collecting tcpdump data yields:

tcpdump -vv -i vm-public (nothing)
tcpdump -vv -i tap0
from guest, ping 8.8.8.8, I see the ping in tcpdump
from guest, ping host, I see the ping in tcpdump
from host, ping guest, I get host is down error
I also see ARP traffic from the guest

My vm switch has my internal LAN network interface as a member:

vm switch create public
vm switch add public lan

GitHub - churchers/vm-bhyve: Shell based, minimal dependency bhyve manager

Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.

github.com

The networking / routing is hazy to me at best. I don't want the VM to be directly exposed to the Internet as that doesn't seem to make sense. It seems to me that if I want this VM to appear as another host on my LAN, the switch should use the LAN interface which it is.

EDIT:
I rebooted my router (the host running vm-bhyve) and miraculously, it works again, go figure. I restarted to apply a kernel update. I wish I knew why.