Other How to set the queue depth on ctl.conf (ISCSI Target)

[belon_cfy]

My ESXI server default queue length is 128 however seems the FreeBSD ctld(8) default queue depth is 32 , when reading/transferring data to the particular lun, the ESXi will fall back to 32 queue length.

Any idea to increase the default queue depth on FreeBSD ctl.conf? I can't find any option or parameter in ctl.conf manual related to the queue depth.

DEVICE                                PATH/WORLD/PARTITION DQLEN WQLEN ACTV QUED %USD  LOAD   CMDS/s  READS/s WRITES/s MBREAD/s MBWRTN/s DAVG/cmd KAVG/
t10.FREEBSD_storage11_iscsi_vol00_42           -              32     -    0    0    0  0.00   520.32     0.38   495.72     0.01    23.76     1.17     0
t10.FREEBSD_storage11_iscsi_vol01_42           -             128     -    0    0    0  0.00     0.38     0.00     0.00     0.00     0.00     0.67     0
t10.FREEBSD_storage31_iscsi_vol00_56           -             128     -    0    0    0  0.00     0.38     0.00     0.00     0.00     0.00     0.52     0
t10.FREEBSD_storage31_iscsi_vol01_56           -             128     -    0    0    0  0.00     7.06     0.95     5.72     0.00     0.06     0.99     0

 

[mav@]

There is simply no queue depth limit in CTL, so there is no option to set it. Though in most cases I would not recommend to send more then few hundred requests same time due to exponential growth of processing overhead. I have no idea from where limit of 32 came, but AFAIK not from CTL/ctld.

 

[trasz@]

Well, actually there is a queue depth for iSCSI - kind of. It's controlled by the kern.cam.ctl.iscsi.maxcmdsn_delta sysctl, and defaults to 256. Still, no idea where did 32 come from.

 

[mav@]

maxcmdsn_delta sysctl is a number of not acknowledged commands in iSCSI request sending queue. It has very limited relations to the SCSI queue depth.

 

[trasz@]

That's true, and that's why I said "kind of". I have on idea what ESX actually reports as "queue depth", to be honest. Still, it's something that is kind of related and might be useful.