New server build

X

xy16644

Hello

In the next couple of months I want to replace my ageing second hand HP dc5800 desktop (with 4GB RAM) I use to run my FreeBSD 9.1 i386 server. It runs an encrypted ZFS root and has an unencrypted /boot. Both live on an mirrored 2.5" 320GB drives and I have a 3rd single drive I use for backups which is 500GB. I have 3 ZFS pools: bootdir, zroot and backups.

So why do I want a new server? For the following reasons:

  1. GELI is slooooow on this machine. Maybe 20MB/sec tops.
  2. I don't have enough RAM to take full advantage of ZFS
  3. Upgrading from source and recompiling all my ports takes up to two days
  4. I have outgrown i386
  5. Theres not much room in the case to add more drives (its limited)

So the requirements for this new build are as follows:
  1. CPU must support AESNI
  2. Will have 16GB of ECC RAM
  3. I will have use SSD drives for root
  4. FreeBSD 9.1 STABLE amd64 will be used
  5. Must be energy effecient and quiet as the server lives in my lounge!

I have been considering the following hardware:

I guess my first question is, will this motherboard work with FreeBSD 9.1 STABLE? Please don't point me to the Supermicro website...they don't list 9.1 in their list. I was hoping someone who has used Supermicro boards would know if this particular board works with 9.1 :stud From googling it looks like it does work but I wanted to be sure.

I was planning on using the two SSDs in a mirror (with encrypted ZFS using GELI), putting /boot on a separate USB key and storing my backups on the Samsung drive (also encrypted ZFS GELI). Is this a good setup? I only have about 20GB of data on my server so the 120GB SSD drives is plenty. I just wanted to make sure I am configuring this setup with maximum performance? Do I need to even bother with ZIL, L2ARC etc etc?

I assume I'll be able to enable ZFS prefetch with 16GB RAM? On my current system it is disabled.

I'll use the USB drive to hold /boot and the encryption keys. This USB will be used to boot the system. Once booted...it will be removed.

The backup drive will hold daily [cmd=""]zfs send[/cmd] and [cmd=""]tar[/cmd] backups. I will probably keep 30 to 60 days worth of backups as well as 12 monthlys.

Am I on the right track here? I just want to make sure the hardware I have chosen will work with FreeBSD 9.1 STABLE amd64 and that the setup I want to configure will have great performance and be secure. This server must also last me 4-5 years.

Thank you for all your input!! :e

PS: The server runs SSH, Postfix, IMAP, Apache, MySQL, Spamassasin, ClamAV, Roundcube and various monitoring tools (like Munin and MRTG). It is connected to an APC UPS.
 
fonz said:
You may want to have a look at that and see what's wrong. GELI isn't supposed to be slooooow.
Click to expand...

I have often read that GELI encryption has an impact on performance of a machine...especially those without AESNI in the CPU.

The drives I currently use are 2.5" 320GB 5400RPM laptop drives...they are nothing fantastic and when I used them in my laptop they were sluggish.

I guess GELI encrypion + slow drives doesn't help with performance ;)
 
You want power efficient and quiet, but then decided on Intel Xeon CPUs? Those two are mutually exclusive. :)
 
That particular Xeon CPU has a rating of 69W...I'd say that's pretty good.

My Intel Core i7-3770 Processor is rated at 77W. I don't even know when my machine is on and its right next to me...very quiet.

The current CPU in the machine is rated at 65W.
 
xy16644 said:
I have often read that GELI encryption has an impact on performance of a machine...
Click to expand...
I've read reports both ways: some say GELI incurs a significant performance penalty, others say it hardly matters. My own experience is that it doesn't matter a whole lot, but it may depend on your hardware. For example, I probably wouldn't be using GELI on my netbook if it didn't have a crypto chip.

xy16644 said:
I guess GELI encrypion + slow drives doesn't help with performance ;)
Click to expand...
Slow drives don't help anyway, but the actual encryption part of GELI is more CPU-bound than I/O-bound if I'm not mistaken.

In any case, GELI might slow things down a bit, that's pretty much intrinsic, and the amount may depend on your hardware, but "slooooow" (as you put it) just doesn't sound right.
 
Maybe I'm exaggerating a bit when it comes to performance and GELI. Let me explain a bit more.

Normal day to day use of the server (as an end user) is fine. Logging into the server and doing most admin type tasks are fine too. But when it comes to installing/upgrading ports and/or upgrading the operating system from source it then seems to take much longer than when I ran UFS (unencrypted). I wouldn't say the machine is ever unresponsive.

Either way, I'd like to replace the machine with something good :e
 
If you have AES-NI support, GELI (or any other AES related task that is accelerated with hardware support) should have pretty much zero performance impact (yes, there is some of course, but disk is way too slow to cause much).

Compiling software should be a lot quicker than without AES hardware support as not only will the storage no longer be bottlenecked - the CPU that was being used for encryption will be available to compile with :D
 
Even with AESNI, I'll bet GELI sucks the life out of those SSDs. We really need to get HDD password support into camcontrol(8). That would be adequate for many situations, and has no speed penalty.
 
I have just built a server with a Xeon E3-1230 v2 CPU on an Asus P8B-E/4L board with 16GB DDR3-1600 ECC, powered by a Thermaltake Berlin 630W PSU. It runs FreeBSD9.1 amd64 with root on ZFS and I am very satisfied. I do not use GELI.
 
wblock@ said:
Even with AESNI, I'll bet GELI sucks the life out of those SSDs. We really need to get HDD password support into camcontrol(8). That would be adequate for many situations, and has no speed penalty.
Click to expand...

I'd be interested to hear what other forum members experience has been with this particular Xeon CPU with AESNI and an SSD drive. I can't believe GELI + SSD + Xeon (with AESNI) is still slow? Say it ain't so :e

My current mirrored setup with GELI gets roughly 20MB/sec. I was hoping with an SSD and AESNI that the speeds would be substantially higher than that...
 
Hi all,

I have spent some more time researching the hardware I want to put into my new FreeBSD server and have made a few changes to my original list as follows:

  • Intel Xeon E3-1230 V2 Ivy Bridge Retail
  • Supermicro X9SCM-iiF Intel C204 Socket 1155 Motherboard
  • Crucial CT2KIT102472BD160B 32GB (4x 8GB) Memory Kit
  • Samsung 256GB 840 Pro Series Basic SSD 7mm UltraSlim x 2
  • Samsung 2TB 5400RPM SATA hard drive
  • Seasonic SS-450RM 450W Modular Power Supply DC-DC 80+ GOLD (PSU)
  • Fractal Design Define Mini mATX Case No PSU

From what I have read the Supermicro works better with FreeBSD than the X9SCM-F-O due to the model of the onboard Intel NICs.

I changed the RAM (and increased it to 32 GB) as from what I have read Crucial RAM works better with the Supermicro boards than the Kingston ValueRAM. The SSDs have been increased to 256 GB capacity and I have chosen a PSU with a fan to help keep things cool(er).

I plan on using dedup on the SSDs. Yeah, I know this requires much RAM (hence the 32 GB) but the pool will only be about 256 GB in size. I *may* add another pool of 300 GB later on so that I have about 556 GB in total of capacity but the server will still have more than enough RAM to run dedup (I've read you need 20 GB RAM per 1 TB of space).

Anyway, appreciate any comments or feedback! ;)
 
Anyone? Just curious if anyone has any experience with this particular motherboard and FreeBSD 9.1 amd64 before placing the order next week.

Thank you! ;)
 
Does anyone know if the Supermicro X10SL7-F motherboard works with FreeBSD 9.2 amd64? I am interested to know if the onboard LAN ports work (and IPMI) as well as the onboard SAS/SATA ports? From what I have read you can use a SATA drive connected to a SAS port, is this correct?

Thanks!
 
Re:

phoenix said:
You want power efficient and quiet, but then decided on Intel Xeon CPUs? Those two are mutually exclusive. :)
Click to expand...
Since this thread has reawakened, I figured I'd reply to this with some recent data.

I have a Dell PowerEdge R710 system with dual X5680 CPUs (pretty much the top of the 5600 range), 48 GB of buffered ECC RAM, 6 x 300 GB 15K SAS drives, RAID controller with BBU, 10 Gb Ethernet, etc. and it consumes less than 240 W on average (measured at the P/S AC inputs). A 24-job parallel build of kernel + world will spike it up to about 275 W. Not bad at all.
 
@xy16644, did you build this server?

My AMD AM3+ server is unreliable (I believe it to be dry joints on the motherboard) so I am looking for a new motherboard and CPU (I have 8GB of Crucial ECC RAM). I would replace the motherboard but would prefer the board to have SATA 6Gb/s ports and a couple of USB 3.0.

I see Intel have done a low power XEON, the 1220L but it doesn't seem to be available in the UK.
Intel-Xeon-E3-1200-V1-V2-and-rumored-V3-600x334.png


Supply of these types of components isnt great in the UK. Here is one option I found:
Intel Server Board S1200V3RPS which is (amazingly) available from PC World for £145.

There is a nice comparison of these boards here:
http://www.servethehome.com/intel-brand ... rocessors/

Either the above motherboard from Intel or the Supermicro board would suffice but Id prefer to keep costs down and use a cheaper CPU, but ECC support for Intel I3's seems hit-and-miss.
 
Last edited by a moderator:
embeddedbob said:
xy16644, did you build this server?
Click to expand...

I did :beergrin I ended up with the following for my FreeBSD server build:

  1. Supermicro X10SL7-F motherboard
  2. Intel Xeon E3-1230 V3 Haswell
  3. 16GB Crucial 1600 ECC 1.35v RAM
  4. Dual Samsung 840 Pro 128GB SSD
  5. 550W Seasonic G-550 Modular PSU
  6. 3 x 120mm BeQuiet case fans
  7. 1 x 140mm BeQuiet case fan
  8. BeQuiet ShadowRock Pro CPU cooler
  9. Fractal Design Mini case
  10. IcyDock Tough Armor MB994SP-4SB-1 (holds 4 SATA 2.5" disks in a single 5.25" bay)

I must say, I am blown away with how awesome the Supermicro board is with FreeBSD! Everything just works with this board. The speed is amazing from the little that I have used it. It's still early days however but I have seen GELI speeds ranging from 140-440 MB/sec. I have installed FreeBSD 9.2-RELEASE, 9.2-STABLE and 10.0-RC3 so far.

The only issue I have had so far (and this has nothing to do with FreeBSD) are the fan controllers on the motherboard. For some reason the fans didn't like the Supermicro onboard fan controllers so the fans would fluctuate in RPM...every 15 seconds the fans would run at full RPM for a few seconds and then drop back down. Even when the fans RPM speed was reduced it was still noisy. The fan speed options (that you set in the IPMI interface) are a joke. You can choose between loud and really loud! Luckily the problem was easily fixed with a Bitfenix Recon fan controller and it is now so quiet I have to check that the machine is on!

Did I mention how amazing IPMI is? Its GREAT!

The Supermicro board I bought has 14 SATA ports in total. 8 are 6Gb/s SAS ports which accept SATA connections too. 2 are 6Gb/s Intel SATA ports and the other 4 SATA ports run at 3Gb/s/ FYI: I did flash my LSI firmware to run in "IT mode". This removed the RAID functionality (aka IR mode). You really need to run the controller in IT mode if you want to use ZFS as this gives ZFS FULL access to the disks. IR mode may cause issues with ZFS.

There is a USB 3.0 header on the board which I have hooked up to the cases USB 3.0 port. There is also an onboard USB port on the motherboard itself but I don't know what speed this runs at (haven't looked as I won't be using it).

I bought all my kit from Scan and they were great in the end. Any questions feel free to ask.
 
That's great, sounds like a good setup.

Scan would be ideal but they don't seem to stock the components; did you make a request and they ordered them in?

The other reason I was interested in a XEON processors is they have "AES New Instructions" which will help when (or if) native ZFS encryption is introduced.
 
I had to contact them via phone and they ordered the parts that I requested. It was all very easy. It did take two weeks though to get all the parts as it was a custom order but it was worth it in the end. FYI: Just because the parts aren't listed on their website doesn't mean they can't get it.

PM me if you want the guys name and contact number I dealt with at Scan.

The thing I really liked about Supermicro was the fact that they list FreeBSD 9.1 on their website as a supported OS!
 
Processor:

Intel® Xeon® Processor E3-1220L v3
# of Cores 2
# of Threads 4
4M Cache,
1.10 GHz/1.5 GHz
Max TDP 13 W

I'm not sure on price of the V3 (I will find out) but V2 is £144. Low power does appeal as sometimes my server is left on for days. Ive yet to find out how the XEONs clock down to low power states when not in use, and if they turn off the power to cores. If they do all of this then there seems little point to buying a 'low power' XEON. However, I presume they do not drop down below the stated base frequency; not sure why this would be?

Intel® Xeon® Processor E3-1225 v3
# of Cores 4
# of Threads 4
8M Cache,
3.20 GHz/3.6 GHz
Max TDP 84 W
£168.70 Inc VAT

I know some people are using i3 but I still cant find if ECC is offically supported.

Motherboard:

Narrowed down to these choices so I'll email scan and see what they can supply at what cost.

I only need:
4x 6Gb/s SATA
2x USB3
ECC Support

Supermicro X10SL7-F (http://www.servethehome.com/Server-deta ... 3-1200-v3/)
Supermicro X10SLH-F
Supermicro X10SLM-F
Supermicro X10SLM+-F
Intel S1200V3RPL
Intel S1200V3RPM
 
You must log in or register to reply here.
Back
Top