Strange performance issue with bhyve vtnet

Ofloo

Ofloo

Host system is running FreeBSD13.0p4 as well as the guest system. It has gigabit connection to the internet. However not complaining but for some reason when doing speedtest on consoles it's always slower but still not complaining.

system spec:

CPU: Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz (2200.06-MHz K8-class CPU)
real memory = 139586437120 (133120 MB)
avail memory = 133408964608 (127228 MB)

Assigned 2 cores to the bhyve guest and about 2G of ram.

Host:
Code: 
# speedtest
Retrieving speedtest.net configuration...
Testing from Telenet (81.82.215.x)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by edpnet NV (Sint-Niklaas) [85.82 km]: 20.149 ms
Testing download speed................................................................................
Download: 406.04 Mbit/s
Testing upload speed......................................................................................................
Upload: 46.16 Mbit/s

Bit slow but still not complaining.
Code: 
# fastest_pkg
pkg0.tuk.freebsd.org: 431.669 KB/s
pkg0.pkt.freebsd.org: 3717.587 KB/s
pkg0.isc.freebsd.org: 476.832 KB/s
pkg0.nyi.freebsd.org: 728.799 KB/s
pkg0.bme.freebsd.org: 3003.952 KB/s

Fastest:
pkg0.pkt.freebsd.org: 3717.587 KB/s


Guest:

How can it cut down the download speed so much on a guest os?

Code: 
# speedtest
Retrieving speedtest.net configuration...
Testing from Telenet (81.82.215.x)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by edpnet NV (Sint-Niklaas) [85.82 km]: 18.452 ms
Testing download speed................................................................................
Download: 10.40 Mbit/s
Testing upload speed.....................................................................................................
.Upload: 45.20 Mbit/s

This is just horrible

Code: 
# fastest_pkg
pkg0.tuk.freebsd.org: 66.195 KB/s
pkg0.isc.freebsd.org: 74.458 KB/s
pkg0.bme.freebsd.org: 177.416 KB/s
pkg0.nyi.freebsd.org: 93.336 KB/s
pkg0.pkt.freebsd.org: 246.704 KB/s

Network bridge between guests I wouldn't say great but that doesn't seem to be the issue.
Code: 
# iperf -c 10.x.x.11
------------------------------------------------------------
Client connecting to 10.x.x.11, TCP port 5001
TCP window size: 48.8 KByte (default)
------------------------------------------------------------
[  1] local 10.x.x.21 port 29536 connected with 10.x.x.11 port 5001
[ ID] Interval       Transfer     Bandwidth
[  1] 0.00-10.01 sec  1.60 GBytes  1.38 Gbits/sec

network connection from guest to other device in the network

Code: 
# iperf -c 10.x.x.15
------------------------------------------------------------
Client connecting to nephele.ofloo.net, TCP port 5001
TCP window size: 80.8 KByte (default)
------------------------------------------------------------
[  1] local 10.x.x.21 port 32220 connected with 10.x.x.15 port 5001
[ ID] Interval       Transfer     Bandwidth
[  1] 0.00-10.02 sec  4.64 GBytes  3.98 Gbits/sec
 
Ofloo said:
Network bridge between guests I wouldn't say great but that doesn't seem to be the issue.
Click to expand...
I have found bridges and taps are not ideal and reduce speed considerably..
Supposedly FreeBSD 13 contains updated bridge code so I really can't speak to that.

How about using a network device passed-thru? Motherboard has 2 ports. Use one for bhyve.
This works well if you have an upstream device for internet connection.
 
I would say there is either a congestion or a misconfiguration somewhere. And it does not necessarily have anything to do with the guest.

Last time I did run into such a thing - download on my outbound router would work fine, on my backbone still acceptable, but from deeper within my LAN it became horribly slow - it finally turned out that the problem was not at all on my site but on the server machine - that cloud host from which I did the downloading needed to disable TSO. And I figured that only by crawling myself though the tcpdump output captured at various points on the way, and understanding when the traffic did stall instead of flowing.

So this kind of hunting can become a bit laboursome, and you would be better off when testing downloading from a machine that you own and can analyze alongside.

When testing further, I would at first cut out the bridge. Put an IP address onto the tap, and have the host route the packets - then one can see and log them there. Because, when the host itself can receive, but dispatching into the guest it is suddenly slow, one of three parts must be the cause: 1. the tap+bhyve, 2. the bridge, 3. the outbound connection. The bridge can be cut out, and then the other two can be differentially compared. (One could, for instance, check if the problem appears also with UDP, and so on.)
 
You must log in or register to reply here.
Back
Top