PPPoE from inside bhyve guest through VLAN


New Member

Messages: 11

My scenario:
I'm building a home firewall/router on an Intel NUC. Host OS if FreeBSD 12.2 and I installed pfsense in a bhyve guest.
I want to build a "router on a stick".
I have a managed switch and plugged in my DSL modem into a vlan port.
The NUC is connected to a trunk port.

I created 2 bridges, 2 tap devices and a vlan device on the host.
bridge0 should be my LAN connection
bridge1 should be my WAN connection
em0 is my NIC.

LAN: I added em0 and tap0 to bridge0 and assigned an IP ( to bridge0 which gives me connectivity to my LAN
WAN: I created "vlan100" off my em0 and attached it together with tap1 to brdige1

I pass tap0 and tap1 to my bhyve guest. I assign an IP ( of my local network to the tap0 interface and can successfully connect to pfsense.
However I can not establish a PPPoE connection. I could not find much information about it but it seems that you can create a PPPoE connection through a bridge/tap only if the "source" interface is _NOT_ a vlan. Not sure if this is a bug somewhere or just esoteric or ....

The whole VLAN setup in general works because I was able to open the connection on the host machine using pppd.

Furthermore I reverted all the setup:
Connect my modem through an untagged port but configure all the other ports on the switch to be a vlan port. Then I attached em0 to bridge1 and vlan100 to bridge0. Effectively inverting my network setup.
Result: I can instantly create a PPPoE connection from my bhyve guest but for some reason only my NUC host itself can reach pfsense then.
Other members of the network can't access the guest and vice versa. It seems somehow that the packets are not properly forwarded from the host to the bhyve guest but I'm not sure why.

1) Why can't I establish a PPPoE connection through a vlan interface?
2) Why is my bhyve guest not accessible when connected through a vlan interface?

So yea... I hope this is somewhat understandable. It seems it's somewhat esoteric what I try to build but on the other hand I don't think it's too crazy but always one tiny detail is missing it seems :D