GELI performance between AESNI and QAT

[JamesElstone]

Hi All,

Would it be expected that AESNI will out perform an Intel Quick Assist card (8950); I'm seeing poor throughput with AESNI not loaded but QAT is running OK?

Could it be the default zfs() encryption (AES-XTS) is supported by AESNI, and not the Intel QAT 8950?

geli list is showing Crypto: hardware with just the qat() kernel module loaded.

Any ideas?

 

[JamesElstone]

# sysctl -a | grep qat
qat0: <Intel dh895xcc QuickAssist> mem 0xd5080000-0xd50fffff,0xdc840000-0xdc87ffff,0xdc880000-0xdc8bffff irq 48 at device 0.0 numa-domain 0 on pci8
qat0: qat_dev0 started 12 acceleration engines
qat0: FW version: 4.18.0
qat_ocf0: <QAT engine>
        value:  /boot/kernel/qat.ko
        value:  /boot/kernel/qat_api.ko
        value:  /boot/kernel/qat_common.ko
        value:  /boot/kernel/qat_hw.ko
        value:  /boot/kernel/qat_dh895xcc_fw.ko
irq85: qat0:b0:157 @cpu0(domain0): 0
irq86: qat0:b1:159 @cpu0(domain0): 1238176
irq87: qat0:b2:161 @cpu0(domain0): 35
irq88: qat0:b3:163 @cpu0(domain0): 0
irq89: qat0:b4:165 @cpu0(domain0): 0
irq90: qat0:b5:167 @cpu0(domain0): 0
irq91: qat0:b6:169 @cpu0(domain0): 0
irq92: qat0:b7:171 @cpu0(domain0): 0
irq93: qat0:b8:173 @cpu0(domain0): 0
irq94: qat0:b9:175 @cpu0(domain0): 0
irq95: qat0:b10:177 @cpu0(domain0): 0
irq96: qat0:b11:179 @cpu0(domain0): 0
irq97: qat0:b12:181 @cpu0(domain0): 0
irq98: qat0:b13:183 @cpu0(domain0): 0
irq99: qat0:b14:185 @cpu0(domain0): 0
irq100: qat0:b15:187 @cpu0(domain0): 0
irq101: qat0:b16:189 @cpu0(domain0): 0
irq102: qat0:b17:191 @cpu0(domain0): 0
irq103: qat0:b18:193 @cpu0(domain0): 0
irq104: qat0:b19:195 @cpu0(domain0): 0
irq105: qat0:b20:197 @cpu0(domain0): 0
irq106: qat0:b21:199 @cpu0(domain0): 0
irq107: qat0:b22:201 @cpu0(domain0): 0
irq108: qat0:b23:203 @cpu0(domain0): 0
irq109: qat0:b24:205 @cpu0(domain0): 0
irq110: qat0:b25:207 @cpu0(domain0): 0
irq111: qat0:b26:209 @cpu0(domain0): 0
irq112: qat0:b27:211 @cpu0(domain0): 0
irq113: qat0:b28:213 @cpu0(domain0): 0
irq114: qat0:b29:215 @cpu0(domain0): 0
irq115: qat0:b30:217 @cpu0(domain0): 0
irq116: qat0:b31:219 @cpu0(domain0): 0
irq117: qat0:ae:221 @cpu0(domain0): 0
dev.qat_ocf.0.enable: 1
dev.qat_ocf.0.%parent: nexus0
dev.qat_ocf.0.%pnpinfo:
dev.qat_ocf.0.%location:
dev.qat_ocf.0.%driver: qat_ocf
dev.qat_ocf.0.%desc: QAT engine
dev.qat_ocf.%parent:
dev.qat.0.cnv_error:
dev.qat.0.fw_counters:
dev.qat.0.mmp_version: 0.0.1
dev.qat.0.hw_version: 0
dev.qat.0.fw_version: 4.18.0
dev.qat.0.heartbeat: 1
dev.qat.0.heartbeat_failed: 0
dev.qat.0.heartbeat_sent: 4
dev.qat.0.dev_cfg: [GENERAL]
dev.qat.0.num_user_processes: 0
dev.qat.0.cfg_mode: ks
dev.qat.0.cfg_services: sym;dc
dev.qat.0.state: up
dev.qat.0.%domain: 0
dev.qat.0.%parent: pci8
dev.qat.0.%pnpinfo: vendor=0x8086 device=0x0435 subvendor=0x8086 subdevice=0x0000 class=0x0b4000
dev.qat.0.%location: slot=0 function=0 dbsf=pci0:15:0:0
dev.qat.0.%driver: qat
dev.qat.0.%desc: Intel dh895xcc QuickAssist
dev.qat.%parent:

vmstat between file copy on a ZFS encrypted root

# vmstat -i | grep qat
irq86: qat0:b1    1238330    363
irq87: qat0:b2    35              0

# cp /test2 /test4

# vmstat -i | grep qat
irq86: qat0:b1    1371801    400
irq87: qat0:b2    35              0

 

[JamesElstone]

RAW
Transfer rates:
outside: 102400 kbytes in 0.528817 sec = 193640 kbytes/sec
middle: 102400 kbytes in 0.236516 sec = 432952 kbytes/sec
inside: 102400 kbytes in 0.234807 sec = 436103 kbytes/sec

GELI with AESNI only loaded
Transfer rates:
outside: 102400 kbytes in 0.388920 sec = 263293 kbytes/sec
middle: 102400 kbytes in 0.372202 sec = 275119 kbytes/sec
inside: 102400 kbytes in 0.355650 sec = 287924 kbytes/sec

GELI QAT AESNI both loaded
Transfer rates:
outside: 102400 kbytes in 0.611140 sec = 167556 kbytes/sec
middle: 102400 kbytes in 0.578597 sec = 176980 kbytes/sec
inside: 102400 kbytes in 0.577639 sec = 177273 kbytes/sec

GELI QAT only loaded
Transfer rates:
outside: 102400 kbytes in 0.592254 sec = 172899 kbytes/sec
middle: 102400 kbytes in 0.570098 sec = 179618 kbytes/sec
inside: 102400 kbytes in 0.568700 sec = 180060 kbytes/sec