Solved IP traffic from host to netgraph VM invalid

D

dautor

I created a ng_bridge with link0 and link1 connected to my physical devices lower and upper.
I then created a bhyve VM with a netgraph virtio-net device that is connected to link2.
Here is `ngctl dot` to help visualize it:
graph.png

Here is ngctl commands I issued:
Code: 
msg re0: setpromisc 1
msg re0: setautosrc 0
mkpeer re0: bridge lower link0
name re0:lower bnet0
connect re0: bnet0: upper link1

From VM I have access to the outside.
From outside I have access to the VM.
The problem is, host cannot talk with VM.

Tcpdump on guest shows traffic, but it complains about invalid checksum.

Am I doing something wrong?
How do I make it work?
 
I don't know a lot about netgraph , but where did you configure the host ip ? Is it on the Nic or on the bridge ? If it is possible try to migrate your ip to the bridge. Let me know how it goes. I am curious !
 
Host IP is on the interface.
Bridge is not visible outside of netgraph and it is not possible to set an address on it.
Here is more information.

Host:
1704639983344.png

VM:
1704639963069.png

Ping from host to VM, captured on VM:
1704640024889.png
 
With some help from IRC people, I now understand what the problem is.
Hardware checksum offloading is enabled and so the kernel is not inserting valid checksums.

I tried inserting ng_checksum node between bnet0: and v0: but failed to configure it to actually do checksumming.
Temporary solution is to just disable offloading:
Code: 
ifconfig re0 -rxcsum -txcsum -rxcsum6 -txcsum6 -vlanhwcsum -vlanhwtso -tso
 
After thinking more about it, I would say it is supposed to work like that.
There should probably be a paragraph in ng_ether(4) about hardware offloading issues.
Feel free to file a bug if you consider this behavior wrong.

I would file a bug, but I'm not sure for what. I think it should just be mentioned in the man page and maybe have some pointers on how to get around it.
I'm leaning more towards making it work with ng_checksum(4) and making a blogpost about it.

I will again try to make it work with ng_checksum(4) and offloading enabled once I catch some spare time.
 
I found the problem. It was that ng_checksum(4) expects raw IP datagrams by default instead of ethernet.
For anyone interested in the solution, here it is:
graph.png

Code: 
msg re0: setpromisc 1
msg re0: setautosrc 0
mkpeer re0: bridge lower link0
name re0:lower bridge0
connect re0: bridge0: upper link1
mkpeer bridge0: checksum link2 in
name bridge0:link2 csum
mkpeer csum: bridge out link0
msg csum: setdlt 1
name csum:out bridge1

No need to turn of hardware checksum offloading!

After that, I'm adding nodes to bridge1 if they expect a correct checksum or bridge0 if they do not.
I will also try to make ng_macfilter(4) work in place of bridge0.

Ideally, performance should also be measured - however, at the moment I do not care for that and lack time and skill to set up a testing environment for that.
 
You must log in or register to reply here.
Back
Top