Intel Xeon vs. Ryzen

[stratacast1]

Hey guys,

I'm not here to start a flame war, just trying to get some opinions on my upgrade path. I upgraded my desktop with the intent of switching my Ryzen 1600 and its mobo to be a server. However, despite Asus claiming the board supports ECC, they have let me down with false information. So I'm in a pickle. I could ask the community if they have a Ryzen server with ECC and what board they use and I'll consider throwing some extra $$ down if it means I can get that working. Otherwise...what are my prospects with a Xeon that supports DDR4? I'd really like to go used if possible to keep costs down. How would a 4C Xeon from the last few years fare against a Ryzen chip with performance and stability for FreeBSD with bhyve and jails? I don't want to dip into DDR3 gens because I have 32GB of DDR4 ECC ready to go.

 

[Remington]

AFAIK there isn't a server grade motherboard that currently supports Ryzen processor. If you are looking for server grade motherboard and I would suggest you to look into Supermicro or Tyan motherboards with Xeon processors since they're designed to run 24/7 for 5 to 7 years. ASUS is more of a desktop motherboard manufacturer catering to high end gamers and I wouldn't use them as server.

Your best place is to look for used Supermicro or Tyan motherboard on ebay or something similar. These still work very good and many companies retire them after 3 years to avoid downtime when the server or hard drive unexpectedly fails. You get a bargain price for used server.

I have 2 servers running 24/7 for 5 years without any problems. It's time for me to make plans to replace them or keep it as a backup server.

 

[stratacast1]

AFAIK there isn't a server grade motherboard that currently supports Ryzen processor. If you are looking for server grade motherboard and I would suggest you to look into Supermicro or Tyan motherboards with Xeon processors since they're designed to run 24/7 for 5 to 7 years. ASUS is more of a desktop motherboard manufacturer catering to high end gamers and I wouldn't use them as server.

Your best place is to look for used Supermicro or Tyan motherboard on ebay or something similar. These still work very good and many companies retire them after 3 years to avoid downtime when the server or hard drive unexpectedly fails. You get a bargain price for used server.

I have 2 servers running 24/7 for 5 years without any problems. It's time for me to make plans to replace them or keep it as a backup server.

Yeah that's another reason why I'm reconsidering and checking out Xeons again. Part of me trusts the longevity of my motherboard but maybe I'm just fooling myself. I have an ancient X3430 that's gotta go. It's been running great in my possession for 3 years now. Just really wish I could go Ryzen and have something different than Intel parts

 

[Remington]

If you're looking for something other than Intel. I have 2 AMD Opteron with total of 32 cores in one of my server with Tyan motherboard. Epyc is the current successor.

 

[stratacast1]

I would probably end up with EPYC, but that's going a bit extreme, this isn't for anything that requires that much power...so if I want anything my option is sadly Xeon...wish AMD would try to fill this hole in the market.

This could be a good option

HP Z440 Workstation Intel Xeon Quad Core 3.5GHz 8GB DDR4 500GB HD NVIDIA Win 10 | eBay

Find many great new & used options and get the best deals for HP Z440 Workstation Intel Xeon Quad Core 3.5GHz 8GB DDR4 500GB HD NVIDIA Win 10 at the best online prices at eBay! Free shipping for many products!

www.ebay.com

My current box is an HP Proliant and its been very good to me. I'd just swap in my existing drives and memory and plug along since it also supports DDR4

 

[Remington]

I would probably end up with EPYC, but that's going a bit extreme, this isn't for anything that requires that much power...so if I want anything my option is sadly Xeon...wish AMD would try to fill this hole in the market.

This could be a good option

HP Z440 Workstation Intel Xeon Quad Core 3.5GHz 8GB DDR4 500GB HD NVIDIA Win 10 | eBay

Find many great new & used options and get the best deals for HP Z440 Workstation Intel Xeon Quad Core 3.5GHz 8GB DDR4 500GB HD NVIDIA Win 10 at the best online prices at eBay! Free shipping for many products!

www.ebay.com

My current box is an HP Proliant and its been very good to me. I'd just swap in my existing drives and memory and plug along since it also supports DDR4

It looks good.

 

[stratacast1]

Might just do that then...kinda defeated the purpose of a few pieces of hardware I purchased but maybe I can piece it all together and make back a few hundred on the deal...

 

[toorski]

Purchased used from craigslist for $300 - 2 years ago
This is my email server (laptop with no-ECC RAM) - hosting smtpd and imap daemons in jail.

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

   Module     Error        RunTime      MFLOPS
                            (usec)
     1      4.0146e-13      0.0067   2093.5276
     2     -1.4166e-13      0.0058   1206.1841
     3      4.7184e-14      0.0048   3507.6718
     4     -1.2557e-13      0.0047   3207.7199
     5     -1.3800e-13      0.0085   3417.3477
     6      3.2380e-13      0.0072   4014.9609
     7     -8.4583e-11      0.0134    897.2027
     8      3.4867e-13      0.0098   3059.2490

   Iterations      =  512000000
   NullTime (usec) =     0.0000
   MFLOPS(1)       =  1535.5699
   MFLOPS(2)       =  1779.4692
   MFLOPS(3)       =  2649.7111
   MFLOPS(4)       =  3427.2306

Sometimes I access the laptop over xrdp with freerdp to look at things in Lumina DE there, if I rly need to see it in GUI. On-24/7ops.

Purchased used from craigslist for $250 -2 years ago
This is my Xeon server/workstation (ECC RAM) - hosting authoritative master and cashing dns, smtpd, imap, few httpd(s), pgsql, php, phython, java and js dev. frameworks, plus more that I can’t think of ATM, all in various jails. Here, I sit directly front of Lumina DE to experiment, learn and play FreeBSD in GUI. On-24/7 ops.

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

   Module     Error        RunTime      MFLOPS
                            (usec)
     1      4.0146e-13      0.0075   1859.7054
     2     -1.4166e-13      0.0065   1076.6817
     3      4.7184e-14      0.0058   2922.2714
     4     -1.2557e-13      0.0062   2435.3352
     5     -1.3800e-13      0.0089   3259.5171
     6      3.2380e-13      0.0088   3291.2893
     7     -8.4583e-11      0.0151    797.0276
     8      3.4867e-13      0.0120   2504.9222

   Iterations      =  512000000
   NullTime (usec) =     0.0000
   MFLOPS(1)       =  1356.8281
   MFLOPS(2)       =  1559.9871
   MFLOPS(3)       =  2272.5397
   MFLOPS(4)       =  2777.4161

Bottom line, I don’t see much of performance or reliability difference between those 2 systems. As you can see, according to “flops” (floating point operations per second), the laptop’s performance is better than Xeon’s

I’ll build new system with Ryzen for FreeBSD, when I get all the components. I should have all in by the end of this week ready and running FreeBSD – I hope. You can look in there to see my progress with Ryzen and FreeBSD.

My wish list of hardware components for new FreeBSD-12.* build.

I have few hours to decide if those are the best and ultimate components for a new build that I plan to house fresh FreeBSD-12.0 server/workstation in. https://www.newegg.com/p/N82E16811353062?Item=N82E16811353062 https://www.newegg.com/asus-tuf-b450m-plus-gaming/p/N82E16813119141...

forums.freebsd.org

 

[stratacast1]

I've been going over the limits on my box. Nextcloud operations on lots of files or compression/decompression and such take a long time (loooong time), I'm streaming with Plex, backups to the disks, web servers, ssh proxy, other file storage/syncing, and I plan on moving my VPN over to it so I can turn one of my boxes into something else...in addition to some other projects that are in the works that will be set up to run on this.

My Xeon is an X3430 and I can't even utilize my SSDs since everything is SATA II. There is a big difference in performance with what I do even between boxes 3 years older than what I have. I know from having worked with them. My demands have grown from it being my own little thing to sharing with more folks. Reliability isn't my issue, this 10 year old box is ol' faithful. But I need more horsepower

 

[Remington]

I’ll build new system with Ryzen for FreeBSD, when I get all the components. I should have all in by the end of this week ready and running FreeBSD – I hope. You can look in there to see my progress with Ryzen and FreeBSD.

My wish list of hardware components for new FreeBSD-12.* build.

I have few hours to decide if those are the best and ultimate components for a new build that I plan to house fresh FreeBSD-12.0 server/workstation in. https://www.newegg.com/p/N82E16811353062?Item=N82E16811353062 https://www.newegg.com/asus-tuf-b450m-plus-gaming/p/N82E16813119141...

forums.freebsd.org

That's kinda overkill for a server and it doesn't support ECC so that's a no-no. It might be okay for experiment or desktop uses but in the long run data will become corrupted if run 24/7. Is that something you can afford to lose?

 

[Phishfry]

I say buy a new SuperMicro X10 board that will take your DDR4 and buy a used V3 LGA2011.(V4 chips have held their price)

Supermicro X10SRL-F Motherboard S-2011 for E5-2600/1600 v3

Motherboard ATX Intel C612 Express Chipset Socket R3 (LGA 2011) for Single Intel Xeon E5-2600/1600 v3 family processors - Supports up to 512GB DDR4 ECC LRDIMM (or) up to 256GB DDR4 ECC RDIMM (8x 288-pin DDR4 DIMM sockets) - On-board : SATA3 (6Gbps) via C612 controller (RAID),Integrated IPMI 2.0...

www.wiredzone.com

Intel Xeon E5-2608L V3 SR21P 2.00GHZ CPU *km | eBay

Find many great new & used options and get the best deals for Intel Xeon E5-2608L V3 SR21P 2.00GHZ CPU *km at the best online prices at eBay! Free shipping for many products!

www.ebay.com

 

[Phishfry]

I really prefer the LV (Low Voltge) suffixed LGA2011 chips.
They have been around since LGA2011 V1.
The same chip monikers are used among all 4 generations.
For example 2630LV* has been a low core count with higher clock speed than most LV chips.
The Mac daddy is 2650LV* with the most cores in a low power envelope. They do hold thier value.
I have low core count 2608LV3 and they are pretty affordable.
The 16xx series was very unimpressive to me unless high clock speed is needed for single threaded apps.

Intel Xeon 8 Core E5-2618LV3 SR200 2.3GHZ 20MB 8 GT/S LGA 2011 V3 CPU Processor | eBay

Find many great new & used options and get the best deals for Intel Xeon 8 Core E5-2618LV3 SR200 2.3GHZ 20MB 8 GT/S LGA 2011 V3 CPU Processor at the best online prices at eBay! Free shipping for many products!

www.ebay.com

To me I don't like buying used motherboards because the sockets are so flimsy. One wrong move and it is dead.
Too many ham fisted ebayer's out there. Repairing sockets fingers is no joke. You better have a nice magnifier.
.
LV Cheat sheet
2630L=Sandy Bridge
2630LV2=IvyBridge
2630LV3=Haswell
2630LV4=Broadwell

SuperMicro boards:
V1 and V2 chips will all work in same board (C602 Chipset)
V3 and V4 chips will all work in the same board (C612 Chipset)

The problem you find with new boards is sometimes they ship with older BIOS that only supports V1 CPU but will support V2 CPU's with a BIOS update. Same with V3 boards.
Stale motherboard inventory may mean you need a V3 CPU to upgrade the BIOS to take V4 chips.
You will see many NewEgg complaints that 'board does not work' when they are not flashed with newer BIOS.
The SuperMicro boards have the ability to upgrade the BIOS without a CPU via the BMC webpage but it is an add-on option that SuperMicro wants to charge you for. Does not work without a serial number. Bahhhh to them.

 

[Phishfry]

Let me give the skinny on HP Xeon boxes.
Motherboard uses custom 18 pin power supply header and is non-standard gear.
I think you can do better.

Notice the connectors from this pic:

HP Z440 Workstation Motherboard LGA 2011 Ddr4 710324-002 761514-0 for sale online | eBay

Find many great new & used options and get the best deals for HP Z440 Workstation Motherboard LGA 2011 Ddr4 710324-002 761514-0 at the best online prices at eBay! Free shipping for many products!

www.ebay.com

 

[Phishfry]

I think you can add the Intel S2600 series motherboards in your search too.
The part numbers are screwy so you need to look hard at the details.

Been wanting to build a ITX virt hotrod but I am all in on EATX these days.

ASRock Rack EPC612D4I Mini ITX Server Motherboard LGA 2011 R3 Intel C612 for sale online | eBay

Find many great new & used options and get the best deals for ASRock Rack EPC612D4I Mini ITX Server Motherboard LGA 2011 R3 Intel C612 at the best online prices at eBay! Free shipping for many products!

www.ebay.com

Would you trust Asrock Rack (their server brand) in a 24/7 rig?

 

[Phishfry]

The Intel E5-2695 and 2697 series are the real fast ones with lots of cores. They do burn around 135 watts.
Which is somewhat linear. The LV chips burn around 70W and have around half the benchmarking performance.

 

[Remington]

Would you trust Asrock Rack (their server brand) in a 24/7 rig?

I wouldn't trust ASUS or Asrock as a primary server. Supermicro and Tyan motherboards have excellent track records of being reliable 24/7. Also Intel have excellent server grade motherboards but its more expensive.

 

[toorski]

The problem you find with new boards is sometimes they ship with older BIOS that only supports V1 CPU but will support V2 CPU's with a BIOS update. Same with V3 boards.
Stale motherboard inventory may mean you need a V3 CPU to upgrade the BIOS to take V4 chips.
You will see many NewEgg complaints that 'board does not work' when they are not flashed with newer BIOS.
The SuperMicro boards have the ability to upgrade the BIOS without a CPU via the BMC webpage but it is an add-on option that SuperMicro wants to charge you for. Does not work without a serial number. Bahhhh to them.

looks like, one can grab new SuperMicro X10 for a reasonable price. BUT then, how much do they want for upgrade of the BIOS - more than selling price of the motherboard? They always have a hook - payless now so we can charge you more later

And then, one has to deal with Ebay sellers, and their game of no shame, to get rest of the so called server grade components, for cheap, that supposedly worked until they arrive at buyer's location. Then, they don't work, are not as described or worse. I gave that game up, along with PayPal, long time ago.

 

[Phishfry]

BIOS downloads are free. It is only this silly 'Broadband Managment Controller' webpage that has an option to flash via BMC.
That will allow a BIOS flash with no CPU installed.
It is some sort of BMC option with serial number needed.

All the normal flashing tools still work fine. I still keep an MS-DOS memstick for flashing.
Supermicro also does a good job of supporting BIOS upgrades.
I just got a fresh BIOS for my X9SRL from Nov2018. Not bad for a 7 year old motherboard.
They also keep their BMC controller updated.

 

[Remington]

And then, one has to deal with Ebay sellers, and their game of no shame, to get rest of the so called server grade components, for cheap, that supposedly worked until they arrive at buyer's location. Then, they don't work, are not as described or worse. I gave that game up, along with PayPal, long time ago.

Ebay and PayPal have toughen up a lot lately protecting the buyers from scammers. If you receive anything wrong or item not as described then you send it back and file a complaint with ebay/paypal to get your money back. I have done it once recently and it was resolved quickly.

 

[Phishfry]

I wouldn't trust ASUS or Asrock as a primary server.

Asus does seem to have a couple of server boards but they use custom backplane risers for PCIe.
I think they are rack boards. I am not very interested in them either. P5-B was a great board!

I have a Gigabyte server line board LGA1151 with E3-1240v6 and it works fine, but their BIOS updates are not nearly as timely as SM.
The Gigabyte LGA2011 boards are not very popular but they are OEM'd to some other brands I have noticed.
Most notably Datto.

If you are going to use bhyve you want cores and memory. Your 32GB will be paltry with a couple of VM's. No offense.
I do feel E5 Xeons are better suited to bhyve than E3 Xeons. Did I mention you want lots of cores?

 

[Phishfry]

looks like, one can grab new SuperMicro X10 for a reasonable price.

Yes the X10 title can be deceiving as well. They have X10 boards with LGA1151 and LGA2011
Same with the X9 Line. They had LGA1150, LGA1151, LGA1356 and LGA2011.

The real key to their system is the next set of letters.

D= Dual Socket and S=Single
X9S** is single socket Sandy/Ivybridge
X10S** is single socket Haswell/Broadwell

The next digit in SuperMicro model numbers is the series. So LGA2011 has a digit for the series. it is 'R'
X9SR* is single socket LGA2011
X10SR* is single socket LGA2011-3
When it comes to dual socket boards they add a letter 'A'
X9DA* is a dual socket LGA2011
X10DA* is a dual socket LGA2011-3
X9DR* is also a dual socket LGA2011
X10DR* is also dual socket LGA2011-3

Then comes the specialty of the board (I have not fully figured these out)
L = For example X9SRL and X10SRL
I = For example X9SRI and X10SRI <<< These seem to have iSCSI booting in the BIOS where others do not.
A = For example X9SRA and X10SRA <<< This is definitely the audio option board.
H = For example X9DRH and X10DRH <<< This option adds onboard SAS controller
G = For example X9DRG and X10DRG <<< These are optimized for graphics cards with 4X dedicated slots full bandwidth for GPU

 

[steveoc64]

I thought AsrockRack had a "server grade AM4" board in their catalog

AsRock Rack X470D4U Micro ATX Server Motherboard AM4 Ryzen - Newegg.com

Buy AsRock Rack X470D4U Micro ATX Server Motherboard AM4 Ryzen & Ryzen 7nm PGA1331 AMD X470 with fast shipping and top-rated customer service. Once you know, you Newegg!

www.newegg.com

Might be worth a look, esp if you already have some AM4 CPUs floating around ?

My (short) experience with FreeBSD on Ryzen is a bit of a mixed bag.
- Getting so much processing power and cores, for the money, and relatively low power consumption == good
- Getting bhyve failing catastrophically trying to boot - Ubuntu, Dragonfly, Win10 == not so good
- I have no problems on this machine loading FreeBSD in a bhyve under freebsd mind you.
- Support the market disrupter and all that == good

Ive had no problems getting bhyve running on i3 / i5 / Xeon 1151 ... but zero luck with Ryzen so far. Hopefully its a bad batch of hardware ?

 

[rigoletto@]

That probably is a board where you can use ECC memory but without actual ECC support.

You could have a look on some Cavium ThunderX (v1), and FreeBSD is supposedly (I don't have one to assure) to be quite stable on ARMv8. Also, IIRC, the ThunderX (V1) isn't affected by Spectre. Or the Raptor Blackbird (POWER9), not so stable I think but should be OK for a home server - and fun.

[EDIT]

Some FreeBSD dev (I don't remember who) got one of those dual socket Raptor and is running FreeBSD on it.

[EDIT]

Oh, there is Ampere eMAG ARMv8 processors too. See PR 237055.

 

[Phishfry]

Here is the Asus AMD server board I was referencing above.
The ONLY reason I considered it, was because it one of the only serverboards to take coreboot.
G34 socket is unknown to me. Throw in those funky PCIe sockets which need a custom riser and I was lost.

ASUS KGPE-D16, Socket G34, AMD (90-MSVD01-G0UAY00Z) Motherboard for sale online | eBay

Find many great new & used options and get the best deals for ASUS KGPE-D16, Socket G34, AMD (90-MSVD01-G0UAY00Z) Motherboard at the best online prices at eBay! Free shipping for many products!

www.ebay.com

I think system76 used these in their open source hardware rack offering. Was pretty darn expensive too.

That probably is a board where you can use ECC memory but without actual ECC support.

I see that in some of my hardware. APU1/2/3 uses ECC soldered onboard but problems implementing it with coreboot.
Probably some patented alogos used...

I bought a whole bunch of DDR3 4GB ECC modules for bulk price and was surprised that they worked on some desktop boards.

 

[steveoc64]

Socket G34 are awesome !

Those old 12 / 16 core opterons are going for peanuts on ebay still, and they (should) perform per-core on a par with Bulldozer (which is pretty good)

Quite a few of those off-brand Chinese board makers are offering brand new G34 boards (with custom heatsink) for under the $100 mark. Been tempted to try one just for fun.