I would like to ask if anyone has knowledge of FreeBSD compatibility with DTLS protocol.
I have a Cisco IOS firewall/router configured as a SSL VPN headend, and used with Cisco AnyConnect client software. The VPN uses TLS and/or DTLS protocol - it will use DTLS if enabled and available, otherwise fall back to TLS.
With the VPN connected from a client PC using DTLS, I can connect and login by SSH client (PuTTY or SecureCRT) to a FreeBSD host behind the firewall, but the session hangs if I do anything that uses more than a smidgen of data. For example, "cat smallfile.log" works, but "cat largefile.log" displays a screen or two of data and hangs. It's also impossible to use a utility like midnight commander. If I disable DTLS in the Cisco headend, the client PC connects using only TLS and SSH sessions to a FreeBSD host through the VPN work as they should.
I have a few FreeBSD (13.2) hosts running, and they all exhibit the same problem with DTLS. The thing is, I also have one Linux host running Suse SLES 11, and SSH sessions to the Suse host through the VPN work perfectly with DTLS enabled. It does not hang or disconnect, midnight commander works, etc.
So what is the issue with FreeBSD and DTLS? It looks to me like a packet fragmentation problem, but it has to be on the FreeBSD side. I cannot blame the Cisco VPN configuration, especially as SSH to Suse Linux works properly.
I tried on one FreeBSD host swapping the sshd server for the one in the openssh-portable package, but this made no difference to the problem.
I could avoid the FreeBSD problem by just disabling DTLS in the VPN headend, but this has a big performance penalty. File transfers between a client PC and Windows file server will run 5x faster with DTLS. Is there anything I can do on FreeBSD to make it work with DTLS?
I have a Cisco IOS firewall/router configured as a SSL VPN headend, and used with Cisco AnyConnect client software. The VPN uses TLS and/or DTLS protocol - it will use DTLS if enabled and available, otherwise fall back to TLS.
With the VPN connected from a client PC using DTLS, I can connect and login by SSH client (PuTTY or SecureCRT) to a FreeBSD host behind the firewall, but the session hangs if I do anything that uses more than a smidgen of data. For example, "cat smallfile.log" works, but "cat largefile.log" displays a screen or two of data and hangs. It's also impossible to use a utility like midnight commander. If I disable DTLS in the Cisco headend, the client PC connects using only TLS and SSH sessions to a FreeBSD host through the VPN work as they should.
I have a few FreeBSD (13.2) hosts running, and they all exhibit the same problem with DTLS. The thing is, I also have one Linux host running Suse SLES 11, and SSH sessions to the Suse host through the VPN work perfectly with DTLS enabled. It does not hang or disconnect, midnight commander works, etc.
So what is the issue with FreeBSD and DTLS? It looks to me like a packet fragmentation problem, but it has to be on the FreeBSD side. I cannot blame the Cisco VPN configuration, especially as SSH to Suse Linux works properly.
I tried on one FreeBSD host swapping the sshd server for the one in the openssh-portable package, but this made no difference to the problem.
I could avoid the FreeBSD problem by just disabling DTLS in the VPN headend, but this has a big performance penalty. File transfers between a client PC and Windows file server will run 5x faster with DTLS. Is there anything I can do on FreeBSD to make it work with DTLS?