I am usually the answer guy but I have a pile of parts growing for a ZFS server and I need some guidance.
NUMA or Single Xeon CPU? I have an Chenbro RM23524 which is a 28" deep behemouth.
I am outfitting it with some NVMe and three LSI SAS2 controllers controlling 8 disks each for 24 disks total. 600GB SAS2 Seagate Enterprise drives.
So LGA2011-v3 motherboards.
I am looking at SuperMicro X10SRi -Single socket ATX board or Dual Asrock Rack C612 EATX board for $80 more.
$250 for a single
$330 for a dualie
Needed-Many PCIe slots with maximum flexibility
I have been a long time Supermicro client.
Recently I didn't hesitate to buy a Gigabyte server board and I really want to give Asrock Rack a try.
Advantages of a Dual CPU Board:
80 Lanes of PCIe compared to 40 lanes of single CPU.
Usually also includes 10G or SAS onboard
Insane amount of RAM possible
Disadvantage of Dual CPU Boards:
NUMA is not that efficient.
(CPU benchmarks back this up. dual cpu rigs score 10,000 single CPU yet only 14,000 in dual CPU configurations.)
Advantages of Single CPU Board.
Small form factor possible
With 20+ cores CPU who needs NUMA?
Smaller power supply less noise.
Disadvantages of Single CPU Board
Slots are not full width. Many only half physical lanes. Only 40 Lanes.
Fewer memory slots.
Many bad comments on Newegg for single 2011v3 boards.
Alot of bad DIMM slot complaints. Too many to be user error.
I am seeing this on many of the X10Sxx single socket boards.
Any Comments or Opinions?
I just bought two E5-2608LV3 for cheap. If I only use one it would not break my heart.
I realize dual CPU for ZFS is stupid. I will probably do Virtualization on it for other tasks too.
NUMA or Single Xeon CPU? I have an Chenbro RM23524 which is a 28" deep behemouth.
I am outfitting it with some NVMe and three LSI SAS2 controllers controlling 8 disks each for 24 disks total. 600GB SAS2 Seagate Enterprise drives.
So LGA2011-v3 motherboards.
I am looking at SuperMicro X10SRi -Single socket ATX board or Dual Asrock Rack C612 EATX board for $80 more.
$250 for a single
$330 for a dualie
Needed-Many PCIe slots with maximum flexibility
I have been a long time Supermicro client.
Recently I didn't hesitate to buy a Gigabyte server board and I really want to give Asrock Rack a try.
Advantages of a Dual CPU Board:
80 Lanes of PCIe compared to 40 lanes of single CPU.
Usually also includes 10G or SAS onboard
Insane amount of RAM possible
Disadvantage of Dual CPU Boards:
NUMA is not that efficient.
(CPU benchmarks back this up. dual cpu rigs score 10,000 single CPU yet only 14,000 in dual CPU configurations.)
Advantages of Single CPU Board.
Small form factor possible
With 20+ cores CPU who needs NUMA?
Smaller power supply less noise.
Disadvantages of Single CPU Board
Slots are not full width. Many only half physical lanes. Only 40 Lanes.
Fewer memory slots.
Many bad comments on Newegg for single 2011v3 boards.
Alot of bad DIMM slot complaints. Too many to be user error.
I am seeing this on many of the X10Sxx single socket boards.
Any Comments or Opinions?
I just bought two E5-2608LV3 for cheap. If I only use one it would not break my heart.
I realize dual CPU for ZFS is stupid. I will probably do Virtualization on it for other tasks too.