Hello, I've got old Xeon 5530 server (will upgrade to 5680). I have plans to use for virtual machines.
Storage configuration
1. 2Tb SATA disk
2. nvme drive for ZIL/L2ARC
fio run from console:
Virtual machine with zvol backend (UFS)
I'm not familiar with virtualization, so I'm not sure is it good or bad or where is bottleneck, but it looks unusable
Maybe, you can give me some hints?
Storage configuration
1. 2Tb SATA disk
2. nvme drive for ZIL/L2ARC
fio run from console:
Code:
abishai@alpha:/test % doas fio --randrepeat=1 --ioengine=sync --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=sync, iodepth=64
fio-3.7
Starting 1 process
Jobs: 1 (f=1): [m(1)][75.0%][r=299MiB/s,w=99.2MiB/s][r=76.5k,w=25.4k IOPS][eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=83097: Sun Jul 29 17:01:52 2018
read: IOPS=71.8k, BW=280MiB/s (294MB/s)(768MiB/2738msec)
bw ( KiB/s): min=138019, max=331536, per=97.63%, avg=280253.00, stdev=81043.62, samples=5
iops : min=34504, max=82884, avg=70063.00, stdev=20261.20, samples=5
write: IOPS=23.0k, BW=93.7MiB/s (98.2MB/s)(256MiB/2738msec)
bw ( KiB/s): min=46131, max=110634, per=97.47%, avg=93481.40, stdev=26987.84, samples=5
iops : min=11532, max=27658, avg=23369.80, stdev=6747.06, samples=5
cpu : usr=27.29%, sys=72.71%, ctx=44, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=196498,65646,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=280MiB/s (294MB/s), 280MiB/s-280MiB/s (294MB/s-294MB/s), io=768MiB (805MB), run=2738-2738msec
WRITE: bw=93.7MiB/s (98.2MB/s), 93.7MiB/s-93.7MiB/s (98.2MB/s-98.2MB/s), io=256MiB (269MB), run=2738-2738msec
Virtual machine with zvol backend (UFS)
Code:
Jul 29 16:54:50: bhyveload -c /dev/nmdm1A -m 4096M -e autoboot_delay=3 -d /dev/zvol/zdata/bhyve/test2/disk0 test2
Jul 29 16:54:53: [bhyve options: -c 1 -m 4096M -AHP -U 885b274f-934d-11e8-ba9a-bcaec547a985]
Jul 29 16:54:53: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/zdata/bhyve/test2/disk0 -s 5:0,virtio-net,tap1,mac=58:9c:fc:02:a1:30]
Jul 29 16:54:53: [bhyve console: -l com1,/dev/nmdm1A]
Code:
abishai@test2:/tmp % doas fio --randrepeat=1 --ioengine=sync --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=sync, iodepth=64
fio-3.7
Starting 1 process
test: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=24.4MiB/s,w=8403KiB/s][r=6258,w=2100 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=628: Sun Jul 29 17:03:07 2018
read: IOPS=4959, BW=19.4MiB/s (20.3MB/s)(768MiB/39620msec)
bw ( KiB/s): min= 3243, max=30898, per=91.38%, avg=18128.09, stdev=7126.81, samples=74
iops : min= 810, max= 7724, avg=4531.61, stdev=1781.73, samples=74
write: IOPS=1656, BW=6628KiB/s (6787kB/s)(256MiB/39620msec)
bw ( KiB/s): min= 1114, max= 9657, per=91.24%, avg=6046.36, stdev=2359.85, samples=74
iops : min= 278, max= 2414, avg=1511.23, stdev=590.00, samples=74
cpu : usr=4.41%, sys=26.93%, ctx=253477, majf=3, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=196498,65646,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=19.4MiB/s (20.3MB/s), 19.4MiB/s-19.4MiB/s (20.3MB/s-20.3MB/s), io=768MiB (805MB), run=39620-39620msec
WRITE: bw=6628KiB/s (6787kB/s), 6628KiB/s-6628KiB/s (6787kB/s-6787kB/s), io=256MiB (269MB), run=39620-39620msec
I'm not familiar with virtualization, so I'm not sure is it good or bad or where is bottleneck, but it looks unusable
Maybe, you can give me some hints?