Building up a 16TB system

[Sfynx]

My reading shows that the 9207arrives in IT mode (meaning pass-through/JBOD mode), which is just what is needed for a ZFS array.

I can't tell yet about the 9211, but this post seems to indicate that it does.

NOTE: But this post indicates that you need to replace the 9211 firmware with an IT version

Any knowledge on these points?

Flashing the firmware is easy enough, the integrated SAS controller on my Supermicro X8SI6-F board (which is essentially an LSI 9211-8i with spoofed PCI ID) came in IR mode and I could just reflash it using LSI's firmware.

 

[dvl@]

Flashing the firmware is easy enough, the integrated SAS controller on my Supermicro X8SI6-F board (which is essentially an LSI 9211-8i with spoofed PCI ID) came in IR mode and I could just reflash it using LSI's firmware.

I'm beginning to like this board. But given that I'm going to have 8x SATA III HDD drives and 2x SSD drives...

Looking at the Newegg page indicates it comes with the cables for all 8 drives connected to the SAS controller.

I take it that's what you are using? SATA drives on the SAS contoller in JBOD?

 

[Sfynx]

I'm beginning to like this board. But given that I'm going to have 8x SATA III HDD drives and 2x SSD drives...

Looking at the Newegg page indicates it comes with the cables for all 8 drives connected to the SAS controller.

I take it that's what you are using? SATA drives on the SAS contoller in JBOD?

Yeah, the thing is hooked up to a 12-port SAS/SATA backplane with three SFF-8087 connected rows of 4 ports.
Two of those go to the SAS controller using SFF-8087 cables, and one row is split among 4 SATA ports using a bracket cable (I figured that if that's gonna be a problem in the future we can always get a SAS expander later)

That leaves 2 SATA ports for boot/cache drives.

It supports a lot less RAM though, max 16 GB unbuffered / 32 GB registered

 

[dvl@]

Mixing brands on the SSDs is a good idea, but you may also want to mix internal controller types. Those are both Sandforce. Plextor, Crucial, some Intel drives, and Samsung use non-Sandforce controllers. I've had good results with Plextor and Intel.

I'm finding that the Sandforce controllers tend to do:

Sustained Sequential Read - Up to 555MB/s
Sustained Sequential Write - Up to 555MB/s

While the Samsung (& others) do:

Max Sequential Read - Up to 520 MB/s
Max Sequential Write - Up to 160 MB/s

It seems only Sandforce has those higher write speeds.

 

[wblock@]

I will accept donations to accept this challenge.

Ha, as will I! The WD Greens or the Seagates appear to be the cheapest. The WD Reds are about 20% more, but it can be a problem finding a place that has them in stock.

 

[dvl@]

Ha, as will I! The WD Greens or the Seagates appear to be the cheapest. The WD Reds are about 20% more, but it can be a problem finding a place that has them in stock.

The Seagates support NCQ...

 

[wblock@]

I'm finding that the Sandforce controllers tend to do:

Sustained Sequential Read - Up to 555MB/s
Sustained Sequential Write - Up to 555MB/s

While the Samsung (& others) do:

Max Sequential Read - Up to 520 MB/s
Max Sequential Write - Up to 160 MB/s

It seems only Sandforce has those higher write speeds.

Yes, but there's a catch. The Sandforce controllers only get those write speeds on compressible data. Other controllers don't use compression. Overall, they're likely faster but not as much as the numbers would suggest. The Sandforce controllers just did not seem fully ready. Then Intel brought out SSDs with their own custom Sandforce firmware this year. If I had to pick a Sandforce SSD, I'd go with those.

 

[dvl@]

That leaves 2 SATA ports for boot/cache drives.

It supports a lot less RAM though, max 16 GB unbuffered / 32 GB registered

Oh. Hmm, I guess I'd need registered. I was planning on 32GB to start....

 

[dvl@]

Ha, as will I! The WD Greens or the Seagates appear to be the cheapest. The WD Reds are about 20% more, but it can be a problem finding a place that has them in stock.

I am becoming more partial to the WD Reds now.

And yes, even Amazon will sell only 5 at at time...

 

[boris_net]

Just curious what kind of transfer rates would you expect from such a build both locally and over the network?

 

[dvl@]

I have absolutely no idea.

But if someone funds me the money, I'll do both a low-end server and a higher end server and compare the differences. I think that'd be a very interesting comparison.