ECC or not? AMD with ECC vs intel without, homserver, consumer parts, zfs.

N

naguz

So, the title says it all, albeit in keywords.

I'm about to upgrade my server currently running in my old desktop - LGA775, E6600 etc. It needs a new PSU and new disks anyway, and a new motherboard, so it's in for a full upgrade.

For PSU I'm thinking about a OCZ MODXSTREAM PRO 500W - overkill, but silent, pretty cheap, and reliable. (AFAIK)
For HDDs I'm thinking 3 3TB WD Reds.

Then there is the question about RAM. It seems to me that when buying RED drives and running ZFS storage for a hopefully painless experience with decent reliability, it would be a bad idea to go with anything but ECC. But, it's not like my data will be residing in RAM most of the time - It's usually sitting on disk, and when read, is transfered out of the system. I've been weighing for and against for a few days here, and can't rally decide. Unbuffered ECC @ 1333Mhz is about the same as normal ddr3 on 1600Mhz, so the price doesn't really come into it, and then it seems silly to go for non-Ecc... But then I can't go with any cheap 1155 mobo and a i3 IB, and must find AMD parts, which I don't know much about. I'm not even sure if I'm debating AMD vs Intel or ECC vs. Non-ECC with myself anymore! Could someone come and make a decision for me or something? :p

I'd like low power usage in idle, but wouldn't mind having a bit of oomph if for example I should decide to transcode videos or whatnot. And I'll probably run GELI on the disks with ZFS on top. I might prefer AMD if for no other reason than not buying into the future monopoly of Intel, all specs and other things equal.

Problem is, I have no clue about AMD performance (or lower intel like i3), and I have only managed to confuse myself even more with my compulsive googling for ECC vs non ECC and AMD and intel performance in a home server scenario. I'd really appreciate any advice and stress relief. :p
 
Use ECC if you can. It depends on motherboard and processor.

If you insist on GELI, get an i5 or something else with AES-NI. GELI will still suck the life out of it, but at least it will run reasonably fast. I bought AMD for years, but they just don't seem competitive with the latest Intel stuff.

If you want low power and cheap, the lowest-end Sandy Bridge Celerons are great. Surprisingly fast, too.

I would pick one of the higher efficiency power supplies, 80+ gold. They are not much more expensive and should be able to pay for the difference in a year or so. Been doing some research on that, and the Rosewill Capstone supplies are supposed to be surprisingly good. The modular 450W is $70 right now, but has been on sale for less. I'm tempted. The other choice would be the Seasonic SSR-360GP. Try to size it so the idle drain is at least 20% of the power supply max to get the best efficiency.
 
I agree completely with wblock, except that I sleep much better if I install ECC memory in the mission critical systems that need to keep running (environmental control system, telephone PBX, etc.). I don't worry so much about my personal desktop/file server. If it crashes I reboot it: minor inconvenience.
 
I´m still paranoid about having ECC everywhere, as much as possible. Bitflips can seriously damage the data you store, and the storage won´t even know, nor care about it. Maybe you´ll end up shelling out a couple of extra bucks but you´ll sleep better knowing your data is safe.

/Sebulon
 
I also make a point of using ECC to try to minimize the risk of data corruption. Some people think the risk isn't worth worrying about, but I prefer not to take chances. I use ZFS for the same reason.

ECC support on cheaper processors is perhaps the only remaining advantage of AMD. There are some problems, however:

1) A lot of motherboards apparently don't support ECC, even if the chipset and CPU do. Asus boards often do, but make sure you check the manual to be sure - it should have a section on configuring ECC.

2) AMD chips use more electricity than the Intel equivalents.

3) It's getting hard to find a good AM3+ motherboard with integrated video and ECC support, which would force you to install a video card. This would probably increase power consumption even more, even if you have no monitor attached. I don't think newer APUs with integrated video support ECC unfortunately.

My own home server (used mainly to store photos and video) is currently using an aging single-core AMD chip with 4GB of RAM. It also has only 2 PCI Express slots, both in use, so I can't add a USB3 card. I've been thinking of upgrading it.

The most likely configuration would be a Xeon 1225v2, 8GB of ECC RAM, and one of these motherboards:

http://www.asus.com/Motherboard/P8C_WS/

http://www.supermicro.com/products/motherboard/Xeon/C216/X9SAE-V.cfm

I could build an AMD system for perhaps $200 less (I'm in Australia), but it would be less powerful and use more electricity. The above server/workstation class motherboards are perhaps also more reliable than a "consumer" grade AMD motherboard.

I eventually decided to make do with my current setup for another year or two.
 
clpollock said:
1) A lot of motherboards apparently don't support ECC, even if the chipset and CPU do. Asus boards often do, but make sure you check the manual to be sure - it should have a section on configuring ECC.

2) AMD chips use more electricity than the Intel equivalents.

3) It's getting hard to find a good AM3+ motherboard with integrated video and ECC support, which would force you to install a video card. This would probably increase power consumption even more, even if you have no monitor attached. I don't think newer APUs with integrated video support ECC unfortunately.
Click to expand...
I have been looking at the Asus M5A78L-M/USB3 which is still availible here (Norway). But the selection is small, that is true (and sad)

@wblock: Is GELI really that CPU intensive? I've used luks on linux wuthout much of a performance hit even on laptops with old ULV-processors. Granted, they never moved that much data around...

Going for a C216-chipset Mobo AND a Xeon breaks my budget quite a bit, sadly. But there is another thread on these forums which says the Celeron G540 Sandy Bridge) actually has EC support if paired with a supported intel server chipset. Does anyone know of other cheap-but-sufficient intel processors this is true for? I can't seem to find a list. I might spend a bit more on a server grade mobo, but not if I have to pair it with an Intel.

That said, I don't need super-low power for economy I just want it to run "cool enough" enough when it's semi-idle, which it will be most of the time. An AMD processor will probably be fine.
 
GELI really does make a big difference, but whether it will be a problem depends on your usage. One thing on my list to try is the built-in encryption on notebook drives. That might be as fast as unencypted, but not all drives support it and I don't know how it can be used on a desktop. And of course it limits drive choice.
 
I've used geli several times, but each time it unexpectedly, after a few weeks, refused the valid passphrase with some terse message. Was good as long as it worked... Several ports have similarities to it and one can use them on part of the filesystem, after practicing enough to understand enough of the usages.
 
naguz said:
I have been looking at the Asus M5A78L-M/USB3 which is still availible Going for a C216-chipset Mobo AND a Xeon breaks my budget quite a bit, sadly. But there is another thread on these forums which says the Celeron G540 Sandy Bridge) actually has EC support if paired with a supported intel server chipset. Does anyone know of other cheap-but-sufficient intel processors this is true for? I can't seem to find a list. I might spend a bit more on a server grade mobo, but not if I have to pair it with an Intel.
Click to expand...
Here in Australia, it looks like you can get a Xeon 1225v2 for around $240, and the cheapest core i3 is around $115. A difference of $125 doesn't seem too bad, considering that the Xeon is a far more powerful processor, and is guaranteed to have ECC support. The difference may be more in Europe, but the Xeon might well be cheaper in the long run, since it will last you longer before you feel the need to upgrade again.
 
J65nko said:
If you plan to use ECC memory http://cr.yp.to/hardware/ecc.html could be an interesting read ;)
Click to expand...
An interesting read indeed.

I also find in extremely annoying that some manufacturers seem to think their motherboard "supports ECC" if it will run with ECC memory installed, even if it doesn't actually enable the ECC functionality.

The Asus AMD motherboards that I've been using for the last 4 years have specific "ECC Configuration" sections with detailed options for setting the level of background scrubbing, etc. so hopefully they really do support ECC.

When I decided to use a socket 2011 Xeon for my latest workstation, ECC support was a lot harder to come by. Although it's expensive in Australia, I decided to use this Supermicro motherboard to remove all doubt:

http://www.supermicro.com/products/motherboard/xeon/c600/x9sra.cfm
 
When I pondered the question of spending a little more on memory to get ECC, our admin showed me a log file from one of the servers which contained the ECC entries. The rate of three days with 4GB looks accurate, I think, and when that bit destroys some data structure which ZFS needs, you may have all the redundancy you want. If the FS is buggered on the high level, you can loose it all. So I spent some bucks more to have that peace of mind.

The Asus AMD boards indeed have a good support for ECC in the BIOS, but do not think that BIOS is meant for anything other than Windows. I rarely had as much problems with ACPI, which has put me a bit off that brand.
 
With all my respect to reliability of ECC I'd note that if the error rate of non-ECC RAM is correct we should disbelieve and discard any data provided by devices with non-ECC RAM...
I'd like to remind you an ancient and still used method of parity check. It's done by the memory module (hardware) and guarantees 100% accuracy in 1 bit alteration.
Do you remember 201 error during IBM/PC computers POST (about 20 years ago)?
Non-ECC can't correct the errors but surely would detect them and lead to system halt.
 
clpollock said:
An interesting read indeed.
When I decided to use a socket 2011 Xeon for my latest workstation, ECC support was a lot harder to come by. Although it's expensive in Australia, I decided to use this Supermicro motherboard to remove all doubt:

http://www.supermicro.com/products/motherboard/xeon/c600/x9sra.cfm
Click to expand...
The cheapest Xeon+Mobo combo availible here, is some 480USD. So it seems I'll go with an Asus M5A78L-M/USB3 with an AMD FX-4100, which also has AES Extensions. This would total at about 225USD. At less than half the price, it seems the logical way to go, especially since I won't have to have a discrete GPU.
 
von_Gaden said:
With all my respect to reliability of ECC I'd note that if the error rate of non-ECC RAM is correct we should disbelieve and discard any data provided by devices with non-ECC RAM...
Click to expand...
Indeed we should, at least take the results with a grain of salt.

Parity might at least halt a system, but then again, is it used at all? According to anecdotal evidence simple parity memory is no longer in wide use, so I would not bet on that to work out. Also, in order to be seen at POST, the memory error must be detected at memory check during boot up, but the times where the BIOS did a real memory check for the full range are gone, aren't they? Today the OS would need to stop and display some kind of error message, but I have not seen them. So if these would be active today, you should have a pretty good chance to see one when you use a standard laptop with, say, 16GB memory during a trans-atlantic flight. Radiation which can cause these flips is a lot harder in that altitudes, you should be almost certain to stop the machine.

Running memtest for the whole flight may show the errors even in absence of ECC/parity, but I'd wager that would get you unwanted attention, staring at a screen where cryptic number patterns are scrolling by.
 
naguz said:
Going for a C216-chipset Mobo AND a Xeon breaks my budget quite a bit, sadly. But there is another thread on these forums which says the Celeron G540 Sandy Bridge) actually has EC support if paired with a supported intel server chipset. Does anyone know of other cheap-but-sufficient intel processors this is true for? I can't seem to find a list. I might spend a bit more on a server grade mobo, but not if I have to pair it with an Intel.

That said, I don't need super-low power for economy I just want it to run "cool enough" enough when it's semi-idle, which it will be most of the time. An AMD processor will probably be fine.
Click to expand...

All C216 chipset boards should support ECC RAM with Pentiums and Celerons, for example Supermicro X9SAE, check the supported CPUs.

The latest Opterons 33XX support ECC RAM, too and matching boards with ECC support might be a bit cheaper (for example the ASUS M5A99FX PRO R2.0 can be had for about 130 Euros), but I couldn't find any actual power uasge figures yet.
 
naguz said:
The cheapest Xeon+Mobo combo availible here, is some 480USD. So it seems I'll go with an Asus M5A78L-M/USB3 with an AMD FX-4100, which also has AES Extensions. This would total at about 225USD. At less than half the price, it seems the logical way to go, especially since I won't have to have a discrete GPU.
Click to expand...

As long as it runs console only. Xorg on FreeBSD does not support those GPUs yet.

Update: that says it has a "ATI Radeon 3000", which actually should work. It's the APUs that are not supported.
 
c_geier said:
All C216 chipset boards should support ECC RAM with Pentiums and Celerons, for example Supermicro X9SAE, check the supported CPUs.

The latest Opterons 33XX support ECC RAM, too and matching boards with ECC support might be a bit cheaper (for example the ASUS M5A99FX PRO R2.0 can be had for about 130 Euros), but I couldn't find any actual power uasge figures yet.
Click to expand...
The Pentiums as Celerons aren't very state-of-the-art though. While I know nothing of their real performance, I'd guess they're generally below an i3. Also, are you sure about that claim?

I could go for an Opteron with the Asus M5A78L-M/USB3 AFAIK, but would it matter much compared to the FX one?

@wblock@: I've never ran anything more graphical than ncurses with FreeBSD. God knows its hard enough to maintain ports with WITHOUT_X11 set... :p
 
I have a Celeron G530 in a server. Kind of a nice little processor, what I hoped the Atom would be. Cheap (under $50, under $35 on sale), low-power, about 85% as fast as an i3 according to cpubenchmark.net. With an SSD or a fast disk, it's great. With an ordinary disk, it's fine.
 
naguz said:
The Pentiums as Celerons aren't very state-of-the-art though. While I know nothing of their real performance, I'd guess they're generally below an i3. Also, are you sure about that claim?
Click to expand...

I haven't tried it myself, but to quote supermicro:

The X9SAE Motherboard Series supports a single Intel® Xeon® E3-1200 v2 series,
Xeon E3-1200 series, Pentium®, and Celeron® processor in an LGA 1155 socket.
Leveraging the features of the Intel C216 chipset, the X9SAE motherboard provides substantial enhancement to system performance high performance gaming
platforms and entry-level workstations.​

note: i3 is not supported.

While Celerons and Pentiums might not have AES-NI and some other features (enabled?), the Ivy Bridge Celerons & Pentiums are of the same generation as the Xeons.
 
The original post is like something I posted somewhere else a year or so ago. Haven't yet built it.

The 2011 xeon processors are quite impressive for idle power and top computing muscle.

But lately I'm been thinking about cores and how I really don't like "hyperthreading". It seems to me a properly coded scheduler with a few extra instructions from the processor could do that same thing as hyperthreading. An instruction such as "in case you stall preload this thread and run it ahead of schedule" and then balance the priorities to compensate. The bulldozer approach seems better to me where two cores sharing a floating point unit, would be even better if the FPU's were in a pool that could be used by all the processors. Nowadays I don't see the FPU being all that necessary, the graphics card has become the new FPU.

But the motherboards seem to be lacking for AMD processors. I prefer all x16 slots with x16 and about 3 8x. I don't need sata 3 for single hard drives, the throughput from the hard drives doesn't max out what current drives are capable of. Sata 3 and usb3 might be nice for an external drive array but I'd give them up for the power savings. But I definitely want ECC. I wouldn't even mind registered memory. It may be a bit slower but if I can put a ton of it in the system I think that would more than make up for it. The truth is the box will sit idle 98% of the time. I don't care if it burns twice as much electricity as other processors when it's maxed out so long as it can keep up. What can I say, I really want one of those 16 core engines.
 
BlueCoder said:
But lately I'm been thinking about cores and how I really don't like "hyperthreading". It seems to me a properly coded scheduler with a few extra instructions from the processor could do that same thing as hyperthreading.
Click to expand...


2 things that will make that hard:

  • The CPU may turn cores off and on as required due to power constraints or to optimise multi-thread vs single thread performance
  • CPUs are designed for existing software. Your funky scheduler will not necessarily speed up existing software
 
Looks like I'm getting my hands on a PowerEdge t310 with an Intel Xeon 3400 series processor and 12gigs of memory - for FREE. :D That's quite a "deal". It needs disks, a GPU and probably some modifications for noise/cooling as it will be standing in my living room.

Anyone knows if it is possible to cram the hardware into a normal ATX case?
 
Many Intel Xeon mainboards use the SSI form factor, which is somewhat larger than ATX, more like Extended ATX.
 
You must log in or register to reply here.
Back
Top