hi all
I spent the last few months building and refining a home server running ZFS V28 on 8.2-STABLE. I previously used Windows Home Server where I enjoyed the ability to store a large number of backups with deduplication. After the decision to drop Drive Extender from the new Home Server, I decided to go for FreeBSD and ZFS which gave me similar features. The ability to extend a pool was important since this allows one to have huge single Samba shares (e.g. a movies directory of 5 TB).
The system has an Asus P8P67 motherboard, with an Intel i5-2500 CPU @ 3.30GHz and 16 GB RAM. I started with 8 GB but must recommend 16 GB if you have a large dedup table (e.g. I backup several different PC's on this server in a directory which has dedup and compression).
The system boots off a 1TB Seagate drive (ST31000524AS JC45) on UFS. The performance specs of this HDD is:
I then created a raidz of 5 x 2TB Seagate drives (ST2000DL003-9VT166 CC32). These drives are 4K aligned (just in case) and I chose the 4 data + 1 parity for optimization reasons I read on this thread.
The performance of the 2TB Seagates are as follows:
Finally, I recently got a Corsair 60 GB SSD (Corsair CSSD-F60GB2-A 2.1b). This is the performance:
I use the SSD on the same controller as a 2GB ZIL and a 40GB L2ARC. I also reserved some space for a swap partition, but am not using it yet. I aligned the partitions according to this thread.
I then used bonnie++ version 1.96 and these are the results:
The dedup was done with SHA256. As can be seen, there is a large performance impact during writing but not so much during reading. It seems L2ARC does have a nice positive impact on read speed.
Any comments?
regards
Malan
I spent the last few months building and refining a home server running ZFS V28 on 8.2-STABLE. I previously used Windows Home Server where I enjoyed the ability to store a large number of backups with deduplication. After the decision to drop Drive Extender from the new Home Server, I decided to go for FreeBSD and ZFS which gave me similar features. The ability to extend a pool was important since this allows one to have huge single Samba shares (e.g. a movies directory of 5 TB).
The system has an Asus P8P67 motherboard, with an Intel i5-2500 CPU @ 3.30GHz and 16 GB RAM. I started with 8 GB but must recommend 16 GB if you have a large dedup table (e.g. I backup several different PC's on this server in a directory which has dedup and compression).
The system boots off a 1TB Seagate drive (ST31000524AS JC45) on UFS. The performance specs of this HDD is:
Code:
# diskinfo -c -t -v ada1
ada1
512 # sectorsize
1000204886016 # mediasize in bytes (931G)
1953525168 # mediasize in sectors
0 # stripesize
0 # stripeoffset
1938021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
5VP7VE4K # Disk ident.
I/O command overhead:
time to read 10MB block 0.082120 sec = 0.004 msec/sector
time to read 20480 sectors 2.698682 sec = 0.132 msec/sector
calculated command overhead = 0.128 msec/sector
Seek times:
Full stroke: 250 iter in 5.819074 sec = 23.276 msec
Half stroke: 250 iter in 3.890680 sec = 15.563 msec
Quarter stroke: 500 iter in 6.128329 sec = 12.257 msec
Short forward: 400 iter in 2.114499 sec = 5.286 msec
Short backward: 400 iter in 2.666716 sec = 6.667 msec
Seq outer: 2048 iter in 0.105536 sec = 0.052 msec
Seq inner: 2048 iter in 0.280450 sec = 0.137 msec
Transfer rates:
outside: 102400 kbytes in 0.820216 sec = 124845 kbytes/sec
middle: 102400 kbytes in 0.971817 sec = 105370 kbytes/sec
inside: 102400 kbytes in 1.592233 sec = 64312 kbytes/sec
I then created a raidz of 5 x 2TB Seagate drives (ST2000DL003-9VT166 CC32). These drives are 4K aligned (just in case) and I chose the 4 data + 1 parity for optimization reasons I read on this thread.
The performance of the 2TB Seagates are as follows:
Code:
# diskinfo -c -t -v ada3
ada3
512 # sectorsize
2000398934016 # mediasize in bytes (1.8T)
3907029168 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
3876021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
5YD4E2K9 # Disk ident.
I/O command overhead:
time to read 10MB block 0.092707 sec = 0.005 msec/sector
time to read 20480 sectors 2.618028 sec = 0.128 msec/sector
calculated command overhead = 0.123 msec/sector
Seek times:
Full stroke: 250 iter in 6.634518 sec = 26.538 msec
Half stroke: 250 iter in 4.936454 sec = 19.746 msec
Quarter stroke: 500 iter in 7.973672 sec = 15.947 msec
Short forward: 400 iter in 2.845359 sec = 7.113 msec
Short backward: 400 iter in 2.793776 sec = 6.984 msec
Seq outer: 2048 iter in 0.123256 sec = 0.060 msec
Seq inner: 2048 iter in 0.108371 sec = 0.053 msec
Transfer rates:
outside: 102400 kbytes in 0.721418 sec = 141943 kbytes/sec
middle: 102400 kbytes in 0.849894 sec = 120486 kbytes/sec
inside: 102400 kbytes in 1.562353 sec = 65542 kbytes/sec
Finally, I recently got a Corsair 60 GB SSD (Corsair CSSD-F60GB2-A 2.1b). This is the performance:
Code:
# diskinfo -c -t -v ada2
ada2
512 # sectorsize
60022480896 # mediasize in bytes (55G)
117231408 # mediasize in sectors
0 # stripesize
0 # stripeoffset
116301 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
1111650335000462002D # Disk ident.
I/O command overhead:
time to read 10MB block 0.042269 sec = 0.002 msec/sector
time to read 20480 sectors 1.353079 sec = 0.066 msec/sector
calculated command overhead = 0.064 msec/sector
Seek times:
Full stroke: 250 iter in 0.029324 sec = 0.117 msec
Half stroke: 250 iter in 0.034488 sec = 0.138 msec
Quarter stroke: 500 iter in 0.105246 sec = 0.210 msec
Short forward: 400 iter in 0.084277 sec = 0.211 msec
Short backward: 400 iter in 0.081533 sec = 0.204 msec
Seq outer: 2048 iter in 0.074574 sec = 0.036 msec
Seq inner: 2048 iter in 0.073741 sec = 0.036 msec
Transfer rates:
outside: 102400 kbytes in 0.453352 sec = 225873 kbytes/sec
middle: 102400 kbytes in 0.490449 sec = 208788 kbytes/sec
inside: 102400 kbytes in 0.409787 sec = 249886 kbytes/sec
I use the SSD on the same controller as a 2GB ZIL and a 40GB L2ARC. I also reserved some space for a swap partition, but am not using it yet. I aligned the partitions according to this thread.
Code:
# gpart show ada2
=> 0 117231408 ada2 BSD (55G)
0 129024 - free - (63M)
129024 4194304 1 freebsd-zfs (2.0G)
4323328 83886080 2 freebsd-zfs (40G)
88209408 27262976 4 freebsd-swap (13G)
115472384 1759024 - free - (858M)
I then used bonnie++ version 1.96 and these are the results:
Code:
Configuration Block Output Rewrite Block Input Random Seeks
MB/s MB/s MB/s /s
UFS 89.8 38.1 97.8 171.2
ZFS 185.5 118.0 275.4 186.7
ZFS+dedup 25.1 15.6 248.8 288.0
ZFS+gzip 282.5 212.1 953.6 450.7
ZFS+gzip+dedup 21.5 16.7 650.7 333.9
ZFS+zil+l2arc 231.8 129.0 256.8 371.2
zfs+zil+l2arc+dedup 21.2 18.3 318.8 568.6
zfs+zil+l2arc+gzip 289.3 233.4 903.4 530.2
zfs+zil+l2arc+gzip+dedup 41.6 25.9 818.9 723.0
The dedup was done with SHA256. As can be seen, there is a large performance impact during writing but not so much during reading. It seems L2ARC does have a nice positive impact on read speed.
Any comments?
regards
Malan