RAID/Mirroring recommendations

[gardner]

I am looking for ideas and best practices for setting up a new box with redundant disks.
The box is a DELL that has two SATA channels and supports "Adaptec CERC SATA RAID". This is said to work with aac(4).

What I want to be able to do is:
(1) rebuild the mirror set in the event that a disk fails, likely on much larger disks that will be current in 4 years time,
(2) extract the data using some other hardware, in the event that the hardware fails, and
(3) maybe take one of the disks, stuff it in a USB drive shell and connect to to a working system and be able to extract the data.

Now the Adaptec CERC has the main point in its favour that it is built into the BIOS, so configuring and rebuilding an array seems simple. It is also nominally supported by FreeBSD so that's great.

On the minus side:
The 2-way Adaptec CERC is a software-only thing, so presumably aac(4) is doing all the real work in software anyway.
Folks say that the Adaptec drivers are flaky, mainly because Adaptec is very closed with their specs
CERC is built into the BIOS, so moving the drives to other hardware that doesn't handle CERC could result in utter lossage.

If I want to use RAID1/mirroring, what is my best way forward with a software-only solution? aac(4) doesn't feel like a particularly safe bet.

 

[tingo]

Is this a trick question?
Try gmirror(8) or zfs.

 

[jb_fvwm2]

I'd also check graid5 or graid3; maybe even gjournalling atop the former coupled with smartmontools. (Others may know of a reason not to, but I know of none[1]). Smartmontools (reallocated sector count) should warn of drive failure provided adequate cooling of
the drives; then hopefully your points #1 and #2 would not be necc. No time to test with hardware though.
[1] unless setup of one wipes the metadata of the other for some reason... but maybe I am thinking of some other (zfs?) scenario.

 

[gardner]

hopefully your points #1 and #2 would not be necc.

I have been running FreeBSD since 2.0.5R and have, at several points, had systems that ran for multiple years of uptime. I've encountered numerous instances of both disk and hardware failure.

I find that failure of either the system or the disk is actually quite inevitable -- one or the other will fail within about 5 years of uptime for sure. When a box has a longish lifetime like this, the chances that exotic controllers or compatible drivers will still be around starts to drop off. Therefore I like to keep things as vanilla as possible.

As I write this I am restoring a filesystem from dumps on a box from the mid '90s with some wacky RAID system that, thankfully, I hadn't used. If I had used the RAID controller, the drives would have had some goofy geometry and I doubt I could have connected them via a USB drive shell. Back in the day, I might have used vinum, and it is said that vinum on-media metadata is compatible to GEOM, but I'd rather not find out different in an emergency.

 

[Terry_Kennedy]

I am looking for ideas and best practices for setting up a new box with redundant disks.
The box is a DELL that has two SATA channels and supports "Adaptec CERC SATA RAID". This is said to work with aac(4).

What I want to be able to do is:
(1) rebuild the mirror set in the event that a disk fails, likely on much larger disks that will be current in 4 years time,
(2) extract the data using some other hardware, in the event that the hardware fails, and
(3) maybe take one of the disks, stuff it in a USB drive shell and connect to to a working system and be able to extract the data.

Most of the RAID and RAID-ish controllers write metadata on the drives, which can make it difficult to recover data by connecting the drive to a generic controller.

Worse, some RAID controllers (you know who you are :r) won't accept drives with metadata written by another controller of the same model. Best case, you'll get a prompt saying something like "Config on disk(s) disagrees with config on controller, which do you want to use?" So even a spare controller may not be sufficient.

If I want to use RAID1/mirroring, what is my best way forward with a software-only solution? aac(4) doesn't feel like a particularly safe bet.

What Dell system is this? Most of the Dell systems that offer a RAID controller as an option have plain SATA ports on the motherboard.

If that's the case, you could remove the RAID controller, use the motherboard ports, and use either gmirror(8) or zfs(8). That would give you the best chances of recovery at some distant point in the future.

 

[gardner]

What Dell system is this? Most of the Dell systems that offer a RAID controller as an option have plain SATA ports on the motherboard.

It is a Precision 670. It has two SATA channels than can be just plain SATA channels, or optionally set up as an aac(4) compatible mirror (or stripe) set. Some experimentation and further reading has convinces me that this is a non-starter.

I will experiment a bit with using gmirror as you and others have suggested. Thank you.

 

[Terry_Kennedy]

I will experiment a bit with using gmirror as you and others have suggested.

The only thing that traditional RAID controllers do better than a gmirror device is transparent booting when the primary (generally lowest-numbered) drive fails. With software mirroring, the normal PC BIOS may not boot without either changing the BIOS config or swapping drive cables. Most BIOS bootstraps will only boot from the first port on the motherboard, or will complain "drive not detected" if you simply un-cable it. This is entirely a BIOS limitation and nothing FreeBSD can fix.

Between gmirror and ZFS, there really isn't much need for hardware RAID controllers on new FreeBSD systems. I'm using a 16-port 3Ware 9650, but in relatively "dumb" mode - it exports each of the 16 drives to FreeBSD as a separate unit. I'm using it because it is a rock-solid controller (there are lots of horror stories here about buggy SATA controllers, even the dumb, non-RAID ones) and has a very nice management interface (sysutils/3dm) which works natively under FreeBSD.

I have a Dell R300 which came with a SAS 6/iR RAID card, which I'm using under FreeBSD. The mpt(4) driver for it doesn't decode any of the asynchronous event notifications it gets from the controller, so the logs are littered with things like:

mpt0: mpt_cam_event: 0x16
mpt0: mpt_cam_event: 0x21
mpt0: mpt_cam_event: 0x21

The only reason I didn't pull the card and simply use the on-board SATA ports is because this system uses an intelligent hot-swap backplane and an oddball cable between the PERC and the backplane.