One-way communication issue with vlan and Netgraph

F

fzfq3m

Hello all, wondering if someone more experienced can lend me a hand to figure out why my VMs don't have access to a vlan network when the tagging is being done by the guest OS instead of router (ingress/egress).

I have a somewhat simple setup
router ---- igb0 ---- pub-br0 ---- vm100
router = self explanatory, (interface is configured to accept only tagged packets)
igb0 = Hardware interface on bhyve host (promiscuous mode enabled and autosrc disbaled)
pub-br0 = ng_bridge connected to igb0:lower via pub-br0:link0 (ngctl mkpeer igb0: bridge lower link0)
vm100 = ng_socket (created automatically by bhyve when using netgraph backend)

With this configuration I have only one-way communication... if I set the vm interface to dhcp then I can see the router getting the dhcp request and sending out the dhcp offer via the correct interface and with the correct vlan tag, but the vm doesn't get the packet and hence keep sending the request. If I set the vm interface to manual and configure the corresponding vlan, I can't not ping the router because the vm doesn't get the arp reply packet from the router even though is being sent (same issue with the dhcp reply)

I have tried capturing via nghook on both upper and orphans hooks of igb0 but nothing shows there... I even tried to add an static arp entry pointing to the router on the vm but no luck either.

No idea why the arp reply offer are not being recieved by the bhyve host (router and host are directly connected)

Any help would be appreciated

Note: If I configure the router to handle the tag/untag then the vm work normally.
 
I don’t have an answer, just trying to think through this…

Is it because igb0 isn’t on the same vlan as the router? I assume router is the defaultrouter from your host and igb0 has no vlan config.

I’d expect the packets to go like this:

- guest tags the packet, sends it out
- host can’t handle it directly, forwards it to router
- router sends a tagged reply
- reply packet comes in on igb0, which doesn’t know anything about vlan and drops (or sends it right back to router where it’s dropped?)

Seems like you’d need a router rule on the host to route packets matching guest IP & vlan to the bridge.

Another alternative may be to create an alias IP on igb0 with vlan set? Assuming that’s possible… then connect that to the bridge, so when a vlan packet comes back from the router it’s a part of the bridge and can make it to the guest.
 
patmaddox said:
I don’t have an answer, just trying to think through this…

Is it because igb0 isn’t on the same vlan as the router? I assume router is the defaultrouter from your host and igb0 has no vlan config.

I’d expect the packets to go like this:

- guest tags the packet, sends it out
- host can’t handle it directly, forwards it to router
- router sends a tagged reply
- reply packet comes in on igb0, which doesn’t know anything about vlan and drops (or sends it right back to router where it’s dropped?)

Seems like you’d need a router rule on the host to route packets matching guest IP & vlan to the bridge.

Another alternative may be to create an alias IP on igb0 with vlan set? Assuming that’s possible… then connect that to the bridge, so when a vlan packet comes back from the router it’s a part of the bridge and can make it to the guest.
Click to expand...
Thank you for answering...
Indeed the router has a virtual interface assigned to said vlan with a dhcp server attached. And indeed everything works if I create a vlan interface in the bhyve host and assign it to the vm... this would be similar scenario as the router doing the vlan tagging.

But my understanding is that setting the nic to promiscuous mode should remove any L2 filtering the nic is doing, so even if the driver/nic knows nothing about the vlan it should forward the packet right? otherwise tools like tcpdump wouldn't work. This is incorrect it seem... I was inspecting igb1 to double check that the interface is in promiscuous mode I noticed there are other options or features my hardware has revealed I might be incorrrect:
flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>

Apparently, the card has hardware-level vlan filtering... I disabled it and I can now see the arp reply/offer from the router on the host interface via tcpdump, but even with that the packets aren't being forward to the vm.

I also found an ild post with similar (if not the same) issue, it has not answer ?
forums.freebsd.org

netgraph ng_bridge: can't ge an IP using DHCP when creating a vlan

I am using a bridge `bridge0` that will contain a native nic interface ixl3 and add a tap interface in it: ifconfig bridge0 create ifconfig bridge0 addm ixl3 addm tap0 ifconfig bridge0 bridge0: flags=8843 metric 0 mtu 1500 ether 58:9c:fc:10:e7:20 id 00:00:00:00:00:00 priority 32768...
forums.freebsd.org forums.freebsd.org
 
You must log in or register to reply here.
Back
Top