12TB+ Home Fileserver Hardware Recommendation

T

TTmonster

I am looking to assemble my first FreeBSD fileserver using ZFS (raidz1) for storing my media collection and individual PC backups on my OS X/Windows home network. The main purpose will simply be storing and sharing files. I'm not interested in live transcoding, but I may use the machine to convert my backed-up DVDs.

So far, I'm considering the following main components, but I am completely open to very different recommendations. Since I've only set up FreeBSD twice before, full compatibility (as well as performance/longevity) is key and something on which I would really appreciate some input.

Since the hard drives are bound to be pretty expensive, I'd like to keep the CPU/Motherboard/RAM/Disk Controller reasonable. Links are from Newegg and B&H Photo Video

* Intel Xeon E3-1230 V2 Ivy Bridge 3.3GHz Server Processor: $239.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16819117286

* ASUS P8B-X LGA 1155 Intel C202 ATX Intel Xeon E3 Server Motherboard: $189.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16813131722

* Kingston 8GB (2 x 4GB) 240-Pin DDR3 SDRAM Unbuffered DDR3 1600 Server Memory KVR16N11S8K2/8: 2 x $43.99 = $87.98
http://www.newegg.com/Product/Product.aspx?Item=N82E16820239434
(According to reviews, the Asus board does not like most 8GB RAM sticks. 16GB seems sufficient to me, but I don't know if ZFS would work better with more.)

* Western Digital 2TB Caviar Black WD2002FAEX SATA Internal Hard Drive: 6 x $149 = $894
http://www.bhphotovideo.com/bnh/controller/home?O=&sku=822504&Q=&is=REG&A=details
(I've seen mixed reviews of these on the forums, but WD drives have always worked well for me and I like the warranty. Other suggestions are welcome.)

For a disk controller to be used in JBOD mode with ZFS (I tried to select from this article), either:

* SUPERMICRO AOC-USAS2-L8i PCI Express SATA / SAS Eight-Port Internal RAID Adapter: $139.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16816101334&nm_mc=KNC-GoogleAdwords&cm_mmc=KNC-GoogleAdwords-_-pla-_-NA-_-NA&gclid=CKOPweipwrQCFUWo4AodwTUARg

or

* Intel RAID Controller Card SATA/SAS PCI-E x8 8internal ports (SASUC8I): $159.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16816117157

I'm not convinced that ECC RAM is necessary for my application, but the Xeon+Asus combo seemed reasonable. If someone can recommend a more affordable alternative based on an Intel i5 or i7 processor, I think that would work as well.

Recommendations for boot disks, a PSU, and a case to hold all of this would also be helpful.

Thanks.
 
Consider Gigabyte rather than Asus for a motherboard. ECC memory would be possible, and not a bad idea.

The 2T WD Black drives have a poor reputation for early failures. The WD Red drives are an alternative for RAID. Depending on when you buy, the 2T Reds may be enough cheaper to afford one or two more drives. Consider also RAIDZ2 or even Z3.

The Seasonic X-series power supplies are very nice. Occasionally they go on sale for half price. The S12II or M12II are also good, and less expensive.
 
Buy several brands for disks (Samsung, Hitachi, Seagate,WD) (or) from different production batches, so chances to have 2 failures in a short time due to manufacturing defects become slim.
 
I would recommend Supermicro for mainboards rather than Asus, and can't comment on Gigabyte at all. While I can recommend Supermicro to begin with due to own experiences with it, it also needs to be noted it is certifying it's hardware to work with FreeBSD 8.3 and 9.0 :D

On the controller, I also recommend LSI, though Supermicro works but requires you to modify the Hardware connector.

BTW. there are some Xeon-E3 mainboard reviews at servethehome.
 
HarryE said:
Buy several brands for disks (Samsung, Hitachi, Seagate,WD) (or) from different production batches, so chances to have 2 failures in a short time due to manufacturing defects become slim.
Click to expand...

This definitely makes sense to me to improve the failure rate, but I've never seen mention of whether this hurts the RAID performance. With a RAID 0 stripe, people seem to want identical hard drives, but I don't know if there is a similar benefit for ZFS raidz1/2.

Does anyone know if putting partitions rather than disks into the RAID is definitely a better approach? Perhaps especially for mixing brands? http://www.freebsddiary.org/zfs-with-gpart.php
 
For the first question; given that zfs runs on top of hardware, I don't think it matters.
As for partitions, as long as they are on different disks it doesn't matter.

In fact, it is a good idea to shave off a couple of megs of each disk (especially different brands), and add them like that. Although zfs will not complain if you add whole disks and they are a bit different size, I prefer the first approach.
 
I recently build a very similar system and did some research and asked around.

As for the board, supermicro has a very good reputation and the X9SCM-iiF specifically has the advantage of having 2 Intel server grade chips (82754L) on it, which are well supported under FreeBSD.

As for the SATA card, many people recommended the IBM M1015 and it is cheap and well supported.
Don't forget to buy SFF-8087 SATA breakout cables if you go for it.

I won't comment on the disks, too many different opinions on those floating around...

Have fun building your box!
 
TTmonster said:
Does anyone know if putting partitions rather than disks into the RAID is definitely a better approach? Perhaps especially for mixing brands? http://www.freebsddiary.org/zfs-with-gpart.php
Click to expand...

Correct, that is the way to go if you are mixing different brand drives. Don't forget to use the gnop() trick for the 4K alignment.

I never mix different brand disks. But I make sure to keep spares. Usually, before a drive gets into production it suffers for at least a few days some stress tests.

However, if you intend to mix different brands then make sure that their specs are similar.
 
gkontos said:
I never mix different brand disks. But I make sure to keep spares.
Click to expand...
Same here. In the past I've discovered that drive manufacturers will ship you a different model of drive if they don't have any of the same model when you return the drive under warranty. This generally happens after a few years. Caused me no end of trouble on some systems that didn't like 3GB/sec SATA drives.

These days, when I build a RAIDzilla II (Pic 1, Pic 2), I order a case of 20 drives and install 16 in the 'zilla. The other 4 ship with the system as spares.

However, if you intend to mix different brands then make sure that their specs are similar.
Click to expand...
True. It isn't as bad as in the "old days", though - remember when SCSI drives had a "spindle sync" signal so they'd all be rotating at the exact same speed and index at the same time? The one good thing about those was that all you had to do was say "spindle sync different firmware versions" and the manufacturer would give you the latest firmware...
 
I strongly recommend getting hotswap bays to place in the 5.25" front wells, and labeling each slot by Bay+Slot (Such as Bay1.3 for Bay 1, slot 3), and using gpt labels reflecting the hard drives position.
This will make it painless to replace drives when they fail; You wouldn't have to power down and/or open chassis to replace a drive.
 
You must log in or register to reply here.
Back
Top