If you're having issues with extremely poor SSH performance since updating to 12, take a look at sshd_config(5)'s IPQoS setting. Starting with OpenSSH 7.8 — which was pulled in with 12.0 (I'm on 12.1) — the defaults changed.
I still haven't figured out who or what combination in the path is to blame, but on a [FreeBSD em(4) interface] —<1Gbe>— [unmanged 1GbE switch] —<1GbE>— [Tomato Wifi AP/Switch (QoS disabled)] —<WiFi>— [Mac laptop] connection, the new defaults imposed a penalty of three orders of magnitude (!!!) dropping a tens-of-MB/s connection to tens-of-KB/s with stalls. Adding
Just putting this out there for the next poor sap pulling what is left of their hair out. This may very well be a Darwin+WiFi+QoS issue, but I'm recording here for others to stumble upon, since it was the FreeBSD 12 (OpenSSH 7.8) upgrade where I ran into the change in IPQoS defaults.
Two additional points for the armchair debuggers (aren't we all?):
I still haven't figured out who or what combination in the path is to blame, but on a [FreeBSD em(4) interface] —<1Gbe>— [unmanged 1GbE switch] —<1GbE>— [Tomato Wifi AP/Switch (QoS disabled)] —<WiFi>— [Mac laptop] connection, the new defaults imposed a penalty of three orders of magnitude (!!!) dropping a tens-of-MB/s connection to tens-of-KB/s with stalls. Adding
IPQoS reliability
to /etc/ssh/sshd_config on the FreeBSD system and restarting sshd restored the old performance.Just putting this out there for the next poor sap pulling what is left of their hair out. This may very well be a Darwin+WiFi+QoS issue, but I'm recording here for others to stumble upon, since it was the FreeBSD 12 (OpenSSH 7.8) upgrade where I ran into the change in IPQoS defaults.
Two additional points for the armchair debuggers (aren't we all?):
- A connection to another FreeBSD 12.1 box traversing the same switches but no WiFi showed no degradation.
- A connection over all the same hardware to a bhyve VM (hosted on the problem FreeBSD 12 box) running Ubuntu 18.04.3 (OpenSSH 7.6) w/
IPQoS=lowdelay throughput
(default) also had the expected (multiple tens-of-MB/s) performance. (I have not tried changing the QoS on the Ubuntu box to the 7.8 defaults (=af21 cs1
) to see if I can re-create the poor performance as of yet.)