Solved FreeBSD 12 + SSH (7.8) = strange (poor) performance due to IPQoS

Eric A. Borisch

Eric A. Borisch

If you're having issues with extremely poor SSH performance since updating to 12, take a look at sshd_config(5)'s IPQoS setting. Starting with OpenSSH 7.8 — which was pulled in with 12.0 (I'm on 12.1) — the defaults changed.

I still haven't figured out who or what combination in the path is to blame, but on a [FreeBSD em(4) interface] —<1Gbe>— [unmanged 1GbE switch] —<1GbE>— [Tomato Wifi AP/Switch (QoS disabled)] —<WiFi>— [Mac laptop] connection, the new defaults imposed a penalty of three orders of magnitude (!!!) dropping a tens-of-MB/s connection to tens-of-KB/s with stalls. Adding IPQoS reliability to /etc/ssh/sshd_config on the FreeBSD system and restarting sshd restored the old performance.

Just putting this out there for the next poor sap pulling what is left of their hair out. This may very well be a Darwin+WiFi+QoS issue, but I'm recording here for others to stumble upon, since it was the FreeBSD 12 (OpenSSH 7.8) upgrade where I ran into the change in IPQoS defaults.

Two additional points for the armchair debuggers (aren't we all?):
  • A connection to another FreeBSD 12.1 box traversing the same switches but no WiFi showed no degradation.
  • A connection over all the same hardware to a bhyve VM (hosted on the problem FreeBSD 12 box) running Ubuntu 18.04.3 (OpenSSH 7.6) w/ IPQoS=lowdelay throughput (default) also had the expected (multiple tens-of-MB/s) performance. (I have not tried changing the QoS on the Ubuntu box to the 7.8 defaults ( =af21 cs1) to see if I can re-create the poor performance as of yet.)
I'm marking this solved since I have a "fix" .... but I'm curious if anyone else has experienced similar issues.
 
I had this issue today after upgrading a Digital Ocean BSD 11.2 server to 12.1. I noticed very slow response in my ssh session, followed by Connection Refused after logging out and trying to reconnect. From that point on, it was just Connection Refused. I'm lucky I stumbled onto your post. Setting IPQoS to "reliability" did the trick.
 
Glad it helped!

Just adding for the record here: it appears the culprit was the old wireless router running Tomato — after replacing it with a newer one I can restore (remove / return to defaults) the IPQoS setting with no appreciable changes in performance. (The old router started flaking out a few days after my first post for all connections, so perhaps this was just a harbinger of things to come...)
 
You must log in or register to reply here.
Back
Top