bhyve zvol slow

[leonardorame]

Hi, I created a Windows VM with vm-bhyve, this VM has two disks, one a raw image configured as "ahci-hd" and the 2nd is a zvol. As I noticed that 2nd disk (disk D: in the Windows vm) is painfully slow, and hangs the VM. I stopped it and I'm converting to raw with this command:

dd if=/dev/zvol/vms/bhyve/windowsvm/disk1 of=/vms/bhyve/windowsvm/disk1.img

The disk is 200gb and I notice the dd process is taking too long, as now it's taking more than 10minutes for 30% of the disk, even with the ZVOL stored on a Pool of two really fast NVME drives.

The zvol was added to the vm with this command:

vm add -d disk -t zvol -s 200G windowsvm

 

[SirDice]

The disk is 200gb and I notice the dd process is taking too long

Yes, you're copying it in 512 byte chunks, that's not going to give you stellar performance. Maybe add bs=4096 or even bigger.

     The dd utility copies the standard input to the standard output.  Input
     data is read and written in 512-byte blocks.

     bs=n     Set both input and output block size to n bytes, superseding the
              ibs and obs operands.  If no conversion values other than
              noerror, notrunc or sync are specified, then each input block is
              copied to the output as a single block without any aggregation
              of short blocks.

 

[atax1a]

OP reports that the performance is also awful in the VM. Perhaps the pool's ashift is wrong for the device type?

 

[leonardorame]

Yes, you're copying it in 512 byte chunks, that's not going to give you stellar performance. Maybe add bs=4096 or even bigger.

     The dd utility copies the standard input to the standard output.  Input
     data is read and written in 512-byte blocks.

     bs=n     Set both input and output block size to n bytes, superseding the
              ibs and obs operands.  If no conversion values other than
              noerror, notrunc or sync are specified, then each input block is
              copied to the output as a single block without any aggregation
              of short blocks.

You are right.

OP reports that the performance is also awful in the VM. Perhaps the pool's ashift is wrong for the device type?

Yes, how can I check that ashift value?

 

[leonardorame]

sudo zpool get ashift
NAME   PROPERTY  VALUE   SOURCE
datos  ashift    0       default
vms    ashift    0       default
zroot  ashift    0       default

 

[atax1a]

sudo zdb | grep ashift:

 

[leonardorame]

sudo zdb|grep ashift
            ashift: 12
            ashift: 12
            ashift: 12

 

[atax1a]

so, that's configured for 4k sectors (2**12); you might have to make the the guest OS aware of that somehow.

 

[leonardorame]

Aparently the issue was related to the network. I installed the virtio-net driver and the problem went away.

 

[sko]

you should use the nvme controller for guest storage, it is by far the fastest out there and outperforms even paravirtualized storage controllers (virtIO) by magnitudes.