NIC tls offload with Mellanox ConnectX-6Dx

G

gopiio

Hello,

I am trying to test the NIC tls offload with Mellanox ConnectX-6Dx card and Nginx. Here is the ifconfig options:

Code: 
RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,NV,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,TXRTLMT,HWRXTSTMP,MEXTPG,VXLAN_HWCSUM,VXLAN_HWTSO,TXTLS_RTLMT

Also, I can see stats empty stats option:
Code: 
dev.mce.0.tls_rx.stats.rx_error: 0
dev.mce.0.tls_rx.stats.rx_resync_err: 0
dev.mce.0.tls_rx.stats.rx_resync_ok: 0
dev.mce.0.tls.stats.tx_error: 0
dev.mce.0.tls.stats.tx_bytes_ooo: 0
dev.mce.0.tls.stats.tx_packets_ooo: 0
dev.mce.0.tls.stats.tx_bytes: 0
dev.mce.0.tls.stats.tx_packets: 0

Nginx config looks like:
Code: 
ssl_conf_command Options KTLS;

Code: 
sysctl -a | grep  kern.ipc.tls.stats.offload_total
kern.ipc.tls.stats.offload_total: 2013

I can see tlsoffload from ktls stats. How do I verify NIC tls offload working OR how do i properly enable it?
 
Set the error log to debug

error_log /var/log/nginx/error.log debug;

And verify if you see BIO_get_ktls_send() and SSL_sendfile() reported in the debug log.
 
yes, ktls is enabled.

Code: 
2024/07/22 22:20:18 [debug] 52038#101363: *1 BIO_get_ktls_send(): 1
2024/07/22 22:20:18 [debug] 52038#101363: *1 SSL_sendfile: 11

ktls is stats counter is also updating:
Code: 
 sysctl -a | grep  kern.ipc.tls.stats.offload_total
kern.ipc.tls.stats.offload_total: 80

As per ktls docs: https://man.freebsd.org/cgi/man.cgi?query=ktls

My issue is, it looks like it is working on TCP_TLS_MODE_SW mode. How do I switch to TCP_TLS_MODE_IFNET option, which is supported by the NIC card?
 
Thank you for your response iRobbery I progressed a bit. TLS1.3 with hw offload works, when I disable LAG -> LACP protocol. It works with LAG -> loadbalancer protocol. When I enable LACP, it fallback to TCP_TLS_MODE_SW. I do not know whether this is limitation on NIC TLS offload.
 
You must log in or register to reply here.
Back
Top