Salu --
I have found some odd breakage while using a KVM guest running FreeBSD 9 as a router for other guests, when the router is using the new virtio driver for networking.
When the default route for a KVM guest is a router which is also a guest on the same KVM host, and which is using the virtio network driver, ping will work between guests on different subnets, but no userland network daemons will respond. If I switch to the e1000 driver on the router, but change nothing else, everything works correctly.
Here is a reproducible test case that will illustrate the problem.
I created three FreeBSD guests on one Linux KVM host. I am using bridged networking on the KVM host, as br0 and br1. One of the guests has two network interfaces and acts as a router between two subnets, as follows:
router1: br0, 192.168.1.1; br1, 192.168.2.1
client1: br0, 192.168.1.100; default route 192.168.1.1
client2: br1, 192.168.2.100; default route 192.168.2.1
I configured virtio network interfaces on all three hosts. I enabled forwarding on router1, but no packet filtering. No NAT is in use.
Result:
Here are the software versions I am using.
KVM host: Ubuntu 11.10
qemu-kvm: 0.14.1
KVM guests: FreeBSD 9
virtio-kmod: 0.228301
Thank you for your time.
I have found some odd breakage while using a KVM guest running FreeBSD 9 as a router for other guests, when the router is using the new virtio driver for networking.
When the default route for a KVM guest is a router which is also a guest on the same KVM host, and which is using the virtio network driver, ping will work between guests on different subnets, but no userland network daemons will respond. If I switch to the e1000 driver on the router, but change nothing else, everything works correctly.
Here is a reproducible test case that will illustrate the problem.
I created three FreeBSD guests on one Linux KVM host. I am using bridged networking on the KVM host, as br0 and br1. One of the guests has two network interfaces and acts as a router between two subnets, as follows:
router1: br0, 192.168.1.1; br1, 192.168.2.1
client1: br0, 192.168.1.100; default route 192.168.1.1
client2: br1, 192.168.2.100; default route 192.168.2.1
I configured virtio network interfaces on all three hosts. I enabled forwarding on router1, but no packet filtering. No NAT is in use.
Result:
- client1 can ping client2, and vice versa.
- ssh works from router1 to client1 and vice versa, and from router1 to client2 and vice versa.
- ssh from client1 to client2 will fail (and vice versa); the client simply hangs indefinitely while trying to connect. tcpdump on client2 will show that the SYN is arriving at client2 port 22, but client2 never replies, nor generates any debug or log output that suggests it ever saw the connection attempt.
- any other userland network service I try (both tcp and udp) will show the same thing -- packets will arrive at client2 from client1, but the daemon seems to never see them. Since ping works, I know the kernel is getting them.
- If I switch back to the e1000 driver on router1, but make no other changes, and make no changes at all to client1 and client2, then ssh will work properly from client1 to client2.
- If I let router1 continue to use virtio interfaces, but move router1 onto a different KVM host -- so that the traffic from client1 to client2 must leave the KVM host via the bridged interface and then return on a different interface - then ssh will work properly from client1 to client2.
Here are the software versions I am using.
KVM host: Ubuntu 11.10
qemu-kvm: 0.14.1
KVM guests: FreeBSD 9
virtio-kmod: 0.228301
Thank you for your time.
