ZFS "zpool create" took too long to complete.

S

scilek

I have just installed FreeBSD 12 on this motherboard:

TUF GAMING X570-PLUS - Tech Specs｜Motherboards｜ASUS Global

Experience superior tech with TUF GAMING X570-PLUS. High-performance AMD gaming motherboard with PCIe 4.0, dual M.2 and Sync RGB lighting.
www.asus.com www.asus.com

My dmesg output is included in the attached file.

As you can see, I have one Kingston NVME disk and seven HP 1TB SSD disks installed on the system. The OS is installed on the NVME disk and I want to create a separate ZFS pool named "tank" on the SSD disks, using this command:

zpool create tank raidz2 /dev/ada0 /dev/ada1 /dev/ada2 /dev/ada3 /dev/ada4 /dev/ada5 /dev/ada6

Contrary to my previous experience with ZFS, the creation of this particular pool took too long, longer than a minute. Is there something I should worry about?
 

Attachments

No idea. ZFS takes it's time, sometimes more, sometimes less. It usually does not tell what it contemplates on, but it usually comes back, and usually the data is intact.

Anyway, I think you will not have so much fun when trying to write that array. I have one of the S700 here, running alongside a Kingston A400. The latter is known to be slow when writing - and the S700 is not so much better. It basically swallows some hundreds of MB, and then it pauses for garbage collection. (I can live with that, the thing was cheap, and my SSD are not supposed to do continuous mass-writing, anyway.) Now if you have seven of these in a RAID, each of them when doing GC will pause the entire RAID.

I suggest You do Your observations by watching closely at gstat -p.
 
raidz has quite a bit of metadata overhead (hence the slightly increased penalty on available/usable space). I very seldomly used raidz, but I also saw long creation times for raidz vdevs and it seems the time it takes to create a raidz vdev increases significantly with the number of disks in one vdev.

As a sidenote to your pool configuration:
Because ZFS can spread load only between vdevs it is usually best practice to have more than one vdev. So instead of stuffing all providers into one big vdev it might be better to use e.g. 2x 3-disk raidz1 + 1 spare.
If you care for maximum performance with redundancy, mirrors are the preferred way to go (3x 2-disk mirror + 1 spare) especially when it comes to IOPs (which are already very restricted if you use SATA disks). They also resilver in a fraction of the time a raidz device needs to resilver (because a lot of metadata has to be reconstructed in raidz).
 
So seven disks, of 3 mirrors, one spare disk, NOT raidz, better for resilver times? Any
disadvantage of not having a raidz or variant? And created using zfs OR zpool commands
in such a way to use vdevs? I know I could google the answer but someone may know
more expertly here, certainly not asking to question the premise but to obtain advice.
 
A RAID-Z2 pool of 7 identical size disks can tolerate 2 faults, and has the capacity of 5 disks.
A combination of two RAID-Z pools of 3 disks each plus one spare can tolerate 1 fault, and has the capacity of 4 disks.
A combination of three 2-disk mirrors plus one spare can tolerate 1 fault, and has the capacity of 3 disks.

Admittedly, the RAID-Z2 pool will be the slowest. And will have the longest resilver time (which affects data durability). And the combination of multiple pools will sometimes (but not always) tolerate multiple faults. But the RAID-Z2 pool has the highest durability and the largest capacity. So this is a performance versus capacity/durability tradeoff.
 
You must log in or register to reply here.
Back
Top