Intel Xeon D 1521 Idle States PWR Management

[Sebastian]

Hello ,

last year I bought the X10SDV-4C-TLN2F which has a an Intel Xeon D 1521 CPU .

It runs fine for a year now , but the power consumption could be better for a home server. I investigated the handbook and found out that Intel Steed Stepping wasn't enabled in the bios. Now I have enabled it and the cpu know downclocks itself ! Yihaaa .

But I thought that this also should improve the power consumption, but that is not the case... from 2400MHz to 800MHz and no difference ? Strange but ok ....

I checked the state of the cpu with

sysctl -a | grep cx
dev.cpu0.cx_method: C1/hlt
dev.cpu0.cx_lowest: C1
dev.cpu0.cx_supported: C1/1/1

It seems that it is not aware of c2 state .

Intel Datasheet

Idle States
Idle States (C-states) are used to save power when the processor is idle. C0 is the operational state, meaning that the CPU is doing useful work. C1 is the first idle state, C2 the second, and so on, where more power saving actions are taken for numerically higher C-states.

Ok , there're talking about the C2 State inside the datasheet , but maybe it is only a copy and paste failure?

My server is running 11.2 and even when booting 12.0 via usb cx_supported remains the same.

Do I have to contact Intel or does anyone know if the Xeon D does not support C2 Power Save State ?


best regards, sebastian

 

[k.jacker]

According to Wikipedia, all idle states but C0 and C1 are optinal.
https://en.wikipedia.org/wiki/Advanced_Configuration_and_Power_Interface

As far as I know it's up to the manufacturer what they want to implement.
I got almost the same motherboard as you got (X10SDV-4C-TLN4F-O) and mine shows support for C2.
I wonder why there is no hw.acpi.cpu.cx_lowest in your output. It looks like that one sets the value for every single CPU.
My hw.acpi.cpu.cx_lowest defaults to C2, but as you see below, if I set it to C1, then every single CPU will also show C1.
I would try to set that to C2, but it should be in the BIOS as well. I could reboot my machine tomorrow and take a look.

# sysctl hw.acpi.cpu.cx_lowest=C1
hw.acpi.cpu.cx_lowest: C2 -> C1

# sysctl -a | grep cx
hw.acpi.cpu.cx_lowest: C1
dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc/bma
dev.cpu.3.cx_usage_counters: 108 0
dev.cpu.3.cx_usage: 100.00% 0.00% last 8058us
dev.cpu.3.cx_lowest: C1
dev.cpu.3.cx_supported: C1/1/1 C2/2/15
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc/bma
dev.cpu.2.cx_usage_counters: 45 0
dev.cpu.2.cx_usage: 100.00% 0.00% last 589us
dev.cpu.2.cx_lowest: C1
dev.cpu.2.cx_supported: C1/1/1 C2/2/15
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc/bma
dev.cpu.1.cx_usage_counters: 89 0
dev.cpu.1.cx_usage: 100.00% 0.00% last 28us
dev.cpu.1.cx_lowest: C1
dev.cpu.1.cx_supported: C1/1/1 C2/2/15
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc/bma
dev.cpu.0.cx_usage_counters: 78 0
dev.cpu.0.cx_usage: 100.00% 0.00% last 3741us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/1 C2/2/15

C3 is definitely not supported as by the manual.

 

[Sebastian]

Thank you for your help sure it would help if you can check the bios settings on your site !

[sharky@freebsd ~] sysctl -a | grep cx
hw.acpi.cpu.cx_lowest: C2
dev.cpu.7.cx_method: C1/hlt
dev.cpu.7.cx_usage_counters: 8810851
dev.cpu.7.cx_usage: 100.00% last 13862us
dev.cpu.7.cx_lowest: C2
dev.cpu.7.cx_supported: C1/1/1
dev.cpu.6.cx_method: C1/hlt
dev.cpu.6.cx_usage_counters: 8826627
dev.cpu.6.cx_usage: 100.00% last 11279us
dev.cpu.6.cx_lowest: C2
dev.cpu.6.cx_supported: C1/1/1
dev.cpu.5.cx_method: C1/hlt
dev.cpu.5.cx_usage_counters: 8709941
dev.cpu.5.cx_usage: 100.00% last 13098us
dev.cpu.5.cx_lowest: C2
dev.cpu.5.cx_supported: C1/1/1
dev.cpu.4.cx_method: C1/hlt
dev.cpu.4.cx_usage_counters: 8662802
dev.cpu.4.cx_usage: 100.00% last 7us
dev.cpu.4.cx_lowest: C2
dev.cpu.4.cx_supported: C1/1/1
dev.cpu.3.cx_method: C1/hlt
dev.cpu.3.cx_usage_counters: 8998341
dev.cpu.3.cx_usage: 100.00% last 2613us
dev.cpu.3.cx_lowest: C2
dev.cpu.3.cx_supported: C1/1/1
dev.cpu.2.cx_method: C1/hlt
dev.cpu.2.cx_usage_counters: 9001753
dev.cpu.2.cx_usage: 100.00% last 820us
dev.cpu.2.cx_lowest: C2
dev.cpu.2.cx_supported: C1/1/1
dev.cpu.1.cx_method: C1/hlt
dev.cpu.1.cx_usage_counters: 9226803
dev.cpu.1.cx_usage: 100.00% last 480us
dev.cpu.1.cx_lowest: C2
dev.cpu.1.cx_supported: C1/1/1
dev.cpu.0.cx_method: C1/hlt
dev.cpu.0.cx_usage_counters: 9040295
dev.cpu.0.cx_usage: 100.00% last 21us
dev.cpu.0.cx_lowest: C2
dev.cpu.0.cx_supported: C1/1/1

My Bios settings .

 

[Sebastian]

 

[k.jacker]

My settings are almost the same as yours.


I had
CPU C6 Report = Disabled
Changed it to what you have (Enabled), but it didn't do anything but reporting that C5 and C6 where supported as well, in dev.cpu.n.cx_supported
dev.cpu.1.cx_lowest wasn't affected and still showed "C2"


But what you have in your last picture, I think I changed that from the defaults.
Here's what I have
Enable CPU HWPM = Native
Enable CPU Autonomous Cstate = Disabled


Maybe it's worth trying that in your BIOS. I have BIOS revision 1.1

 

[Sebastian]

My BIOS is revsion 2.0a , the most recent.

I changed Enable CPU Autonomous Cstate = Disabled -> Enable .

This is the result.

[sharky@freebsd ~]  sysctl -a | grep cx
hw.acpi.cpu.cx_lowest: C2
dev.cpu.7.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.7.cx_usage_counters: 1016641 11483407 0
dev.cpu.7.cx_usage: 8.13% 91.86% 0.00% last 75us
dev.cpu.7.cx_lowest: C2
dev.cpu.7.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.6.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.6.cx_usage_counters: 1048316 11603797 0
dev.cpu.6.cx_usage: 8.28% 91.71% 0.00% last 33us
dev.cpu.6.cx_lowest: C2
dev.cpu.6.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.5.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.5.cx_usage_counters: 1051327 11215187 0
dev.cpu.5.cx_usage: 8.57% 91.42% 0.00% last 202us
dev.cpu.5.cx_lowest: C2
dev.cpu.5.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.4.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.4.cx_usage_counters: 1044267 11103393 0
dev.cpu.4.cx_usage: 8.59% 91.40% 0.00% last 291us
dev.cpu.4.cx_lowest: C2
dev.cpu.4.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.3.cx_usage_counters: 1073366 11691894 0
dev.cpu.3.cx_usage: 8.40% 91.59% 0.00% last 654us
dev.cpu.3.cx_lowest: C2
dev.cpu.3.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.2.cx_usage_counters: 1153078 11632059 0
dev.cpu.2.cx_usage: 9.01% 90.98% 0.00% last 531us
dev.cpu.2.cx_lowest: C2
dev.cpu.2.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.1.cx_usage_counters: 1114139 11972928 0
dev.cpu.1.cx_usage: 8.51% 91.48% 0.00% last 36us
dev.cpu.1.cx_lowest: C2
dev.cpu.1.cx_supported: C1/1/1 C2/2/15 C3/2/41
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.0.cx_usage_counters: 1072847 11541276 0
dev.cpu.0.cx_usage: 8.50% 91.49% 0.00% last 258us
dev.cpu.0.cx_lowest: C2
dev.cpu.0.cx_supported: C1/1/1 C2/2/15 C3/2/41

Now it seems that the os recognizes C2 and C3 ,though C3 is not supported ^^ .

But even with this settings I could not see any improvements in the power consumption.
If I have time during christmas I will unplug all my drives and how much the mainboard consumes without any additional hardware.

 

[k.jacker]

I think It's not possible to noticeable reduce power usage.

When I bought the mb, I expected less power usage. Those Xeon Ds are based on the Intel Broadwell generation, and even my Desktop Broadwell CPU with Iris Pro graphics doesn't consume more that 20w idle with a good psu.

I have never tried without disks, but with 4 x WD Red 2TB, 32GiB RAM and the 4/8 core XeonD, the system consumes 50W idle with the standard PSU that comes with the little Supermicro cube case (I'm not at home, so can be more specific on that).
Even though the Xeon D has a more power hungry subsystem with more PCIe lanes and, at least on mine, 10gbe, 50W feels a bit to much. I wonder how much the IPMI adds to power cunsumption.

Would be interesting to see your power consumption.