Hi all,
Working on a personal homelab project where I'm testing high-throughput telemetry and syslog aggregation inside a Bhyve guest running on FreeBSD 14.1-RELEASE (host has dual Intel X540-AT2 10GbE interfaces).
The guest is attached via standard
The issue:
TCP throughput over iperf3 between external nodes and the Bhyve guest easily saturates 9.2 Gbps with low CPU utilization. However, when streaming high-rate UDP datagrams (>45,000 pps, 512-byte packets),
In /etc/sysctl.conf on both host and guest, I've raised:
If I run the exact same UDP receiver daemon directly on the FreeBSD host (no Bhyve / tap bridge), zero packets are dropped at the same rate.
Has anyone tuned tap(4) ring buffers or Bhyve VirtIO-net queue sizes for high-packet-rate UDP workloads? Would switching to vale(4) (netmap) switch or passing the NIC virtual function directly via PPT/PCI passthrough be the recommended path here?
Thanks!
Working on a personal homelab project where I'm testing high-throughput telemetry and syslog aggregation inside a Bhyve guest running on FreeBSD 14.1-RELEASE (host has dual Intel X540-AT2 10GbE interfaces).
The guest is attached via standard
tap0 bridged to bridge0 ( ifconfig bridge0 addm ix0 addm tap0 up). On the host, PF handles stateful NAT and redirection rules for incoming traffic.The issue:
TCP throughput over iperf3 between external nodes and the Bhyve guest easily saturates 9.2 Gbps with low CPU utilization. However, when streaming high-rate UDP datagrams (>45,000 pps, 512-byte packets),
netstat -s -p udp inside the guest shows a steady ~4-6% packet drop under socket buffer overflows, and the host logs occasional tap0: watchdog timeout or dropped packets on the bridge.In /etc/sysctl.conf on both host and guest, I've raised:
Code:
kern.ipc.maxsockbuf=16777216
net.inet.udp.recvspace=4194304
net.inet.udp.maxdgram=65535
net.link.tap.user_open=1If I run the exact same UDP receiver daemon directly on the FreeBSD host (no Bhyve / tap bridge), zero packets are dropped at the same rate.
Has anyone tuned tap(4) ring buffers or Bhyve VirtIO-net queue sizes for high-packet-rate UDP workloads? Would switching to vale(4) (netmap) switch or passing the NIC virtual function directly via PPT/PCI passthrough be the recommended path here?
Thanks!
