bhyve Bhyve rtt/performance drop

so then i will test:


root@bsd:~ # sysctl net.link.vlan.soft_pad=1
net.link.vlan.soft_pad: 0 -> 1
root@bsd:~ #
will let you know if i can see a diffrence ...
 
Sorry for a short answer – lot of job right now.

If you set an MTU at 1500 on all you devices there is a frame size (packet length for Ethernet) of 1518 bytes there 18 bytes is a overhead (header of 14 bytes and frame check CRC of 4 bytes), you extend that if you go above 1500, witch is not good!

The VLAN standard (IEEE 802.3ac) increase the Ethernet frame size to 1522 bytes from normal 1518 bytes (on devices that support VLAN tag) automatically from an MTU on 1500 – You do not need to do anything! Therefor, do not extend and put anything more on 1500, you destroy the standard and create delays, RTS etc. on your network. You will see that in tcpdump, tshark, Wireshark.

The frame size (if we think on the same thing) is not the same as MTU, frame size can be big but the an MTU should not extend 1500 (you see 1518 or 1522 in tcpdump - you linked to the same thing).

Try 1500 on all device and see. But consider 9000, it will be faster.

You can have internal networks on 9000 on a NIC and then have a separated NIC to the internet (or other network) on 1500 MTU routed (i don’t know you setup so..).
As example; I run some HA-FreeBSD firewalls with 16 VLANs all on 9000 and one leg at MTU 1500 to the internet. Full speed on every leg, no dup pack, RTS etc. But that's firewalls not Bhyve.
 
hmmm now all interfaces are on 1500Byte of MTU but the Latency/RTT is not the expected.
ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=116 time=4.744 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=116 time=2.468 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=116 time=2.492 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=116 time=44.022 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=116 time=2.847 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=116 time=2.375 ms
64 bytes from 8.8.8.8: icmp_seq=6 ttl=116 time=2.362 ms
64 bytes from 8.8.8.8: icmp_seq=7 ttl=116 time=34.148 ms
64 bytes from 8.8.8.8: icmp_seq=8 ttl=116 time=4.340 ms
64 bytes from 8.8.8.8: icmp_seq=9 ttl=116 time=2.729 ms
64 bytes from 8.8.8.8: icmp_seq=10 ttl=116 time=31.785 ms
64 bytes from 8.8.8.8: icmp_seq=11 ttl=116 time=29.939 ms
64 bytes from 8.8.8.8: icmp_seq=12 ttl=116 time=2.556 ms
^C
--- 8.8.8.8 ping statistics ---
13 packets transmitted, 13 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.362/12.831/44.022/15.083 ms

is there any kind of idea if cpu can assigned hard ? (with loader="bhyveload")


is there some knowledge how failures can be showened?



i learned that
bhyvectl --get-stats --vm=pfsense1
shows a lot of information but unfotentely i'm not able to fully understand the values showed:

bhyvectl --get-stats --vm=pfsense1
vcpu0 stats:
vcpu total runtime 365913708424315
number of ticks vcpu was idle 463400685
number of NMIs delivered to vcpu 0
number of ExtINTs delivered to vcpu 0
EOI without any in-service interrupt 0
error interrupts generated by vlapic 0
timer interrupts generated by vlapic 31469078
corrected machine check interrupts generated by vlapic 0
lvts triggered[0] 0
lvts triggered[1] 0
lvts triggered[2] 0
lvts triggered[3] 0
lvts triggered[4] 0
lvts triggered[5] 0
lvts triggered[6] 0
ipis sent to vcpu[0] 0
ipis sent to vcpu[1] 11293677
ipis sent to vcpu[2] 11764067
ipis sent to vcpu[3] 11684350
ipis sent to vcpu[4] 9385610
ipis sent to vcpu[5] 11432715
ipis sent to vcpu[6] 11363206
ipis sent to vcpu[7] 11283473
ipis sent to vcpu[8] 0
ipis sent to vcpu[9] 0
ipis sent to vcpu[10] 0
ipis sent to vcpu[11] 0
ipis sent to vcpu[12] 0
ipis sent to vcpu[13] 0
ipis sent to vcpu[14] 0
ipis sent to vcpu[15] 0
Resident memory 25820254208
Wired memory 25769807872
Number of vpid invalidations saved 0
Number of vpid invalidations done 47502138
vcpu migration across host cpus 47502138
total number of vm exits 4129678177
vm exits due to external interrupt 188824302
number of times hlt was intercepted 87219895
number of times %cr access was intercepted 2
number of times rdmsr was intercepted 839
number of times wrmsr was intercepted 420
number of monitor trap exits 0
number of times pause was intercepted 2327204093
vm exits due to interrupt window opening 34269784
vm exits due to nmi window opening 0
number of times in/out was intercepted 128857985
number of times cpuid was intercepted 3654100
vm exits due to nested page fault 0
vm exits for instruction emulation 1359646756
number of vm exits for unknown reason 0
number of times astpending at exit 21790776
number of times idle requested at exit 358
number of vm exits handled in userspace 3902930347
number of times rendezvous pending at exit 10
number of vm exits due to exceptions 0



happy for every helping hint
 
You must log in or register to reply here.
Back
Top