ZFS Two questions on ZFS HDD to usable capacity and mixing vdev types in a pool

Hi all,

I am testing different options for ZFS pools. I just created a few files to represent 800MB, 400MB, 300MB and 100MB drives using mkfiles. (think of 10 times each and that is my setup with 8TB, 4TB, 3TB and 1TB drives - although I know the manufacturers numbers are not in TB).

So I end up with this:
Code:
root@oz:~/zfstest # ls -lh
total 1152
-rw-------  1 root  wheel   800M Nov 25 23:05 sd0
-rw-------  1 root  wheel   800M Nov 25 23:05 sd1
-rw-------  1 root  wheel   100M Nov 25 23:05 sd10
-rw-------  1 root  wheel   100M Nov 25 23:05 sd11
-rw-------  1 root  wheel   100M Nov 25 23:05 sd12
-rw-------  1 root  wheel   100M Nov 25 23:05 sd13
-rw-------  1 root  wheel   100M Nov 25 23:05 sd14
-rw-------  1 root  wheel   100M Nov 25 23:05 sd15
-rw-------  1 root  wheel   400M Nov 25 23:05 sd2
-rw-------  1 root  wheel   400M Nov 25 23:05 sd3
-rw-------  1 root  wheel   300M Nov 25 23:05 sd4
-rw-------  1 root  wheel   300M Nov 25 23:05 sd5
-rw-------  1 root  wheel   100M Nov 25 23:05 sd6
-rw-------  1 root  wheel   100M Nov 25 23:05 sd7
-rw-------  1 root  wheel   100M Nov 25 23:05 sd8
-rw-------  1 root  wheel   100M Nov 25 23:05 sd9
root@oz:~/zfstest # ls -l
total 1152
-rw-------  1 root  wheel  838860800 Nov 25 23:05 sd0
-rw-------  1 root  wheel  838860800 Nov 25 23:05 sd1
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd10
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd11
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd12
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd13
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd14
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd15
-rw-------  1 root  wheel  419430400 Nov 25 23:05 sd2
-rw-------  1 root  wheel  419430400 Nov 25 23:05 sd3
-rw-------  1 root  wheel  314572800 Nov 25 23:05 sd4
-rw-------  1 root  wheel  314572800 Nov 25 23:05 sd5
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd6
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd7
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd8
-rw-------  1 root  wheel  104857600 Nov 25 23:05 sd9

I add each pair of 800MB, 400MB and 300MB but I consistently get less reported in each mirror created:

Code:
NAME                     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
testzfs                 2.28G   448K  2.28G        -         -     0%     0%  1.00x  ONLINE  -
  mirror                 768M   268K   768M        -         -     0%     0%
    /root/zfstest/sd0       -      -      -        -         -      -      -
    /root/zfstest/sd1       -      -      -        -         -      -      -
  mirror                 384M    92K   384M        -         -     0%     0%
    /root/zfstest/sd2       -      -      -        -         -      -      -
    /root/zfstest/sd3       -      -      -        -         -      -      -
  mirror                 288M    88K   288M        -         -     0%     0%
    /root/zfstest/sd4       -      -      -        -         -      -      -
    /root/zfstest/sd5       -      -      -        -         -      -      -
  raidz2                 896M      0   896M        -         -     0%     0%
    /root/zfstest/sd6       -      -      -        -         -      -      -
    /root/zfstest/sd7       -      -      -        -         -      -      -
    /root/zfstest/sd8       -      -      -        -         -      -      -
    /root/zfstest/sd9       -      -      -        -         -      -      -
    /root/zfstest/sd10      -      -      -        -         -      -      -
    /root/zfstest/sd11      -      -      -        -         -      -      -
    /root/zfstest/sd12      -      -      -        -         -      -      -
    /root/zfstest/sd13      -      -      -        -         -      -      -
    /root/zfstest/sd14      -      -      -        -         -      -      -
    /root/zfstest/sd15      -      -      -        -         -      -      -

With proper 800MB files acting as storage unit, why am I getting only 768MB reported as SIZE. It is consistent across all drives, I would like to understand the delta.

On a separate note, I could force add the raidz2 to the pool made of multiple mirror vdevs but I read the error message. Again if somebody can explain what could go wrong with a raidz2 combined with multiple mirrors, that would be great. I suspect this is generic warning for vdev type mismatch as you cannot lose a vdev otherwise you lose the pool.

Let me know if I have missed anything else on the last point.
 
First question, just for background research: Can you figure out which of the above numbers are in MiB/GiB, and which in MB/GB? Or get everything to report capacities in bytes?

The difference between MB (decimal = 10^6) and MiB (binary = 2^20) is already 4.5%. The problem you are reporting is also about 4-5%. Could be as easy as the that common confusion.

After you are done with that: Can you do some background research on whether ZFS reports capacities after reserving some space for metadata and logs?
 
Back
Top