Power consumption in suspend

Spica · Monday at 9:51 AM

Can anyone explain why my FreeBSD 14.3-Release-P7 workstation consumes more than four times the power (7W) than a similar specced Debian 12 Bookworm box (1.5W) when suspended...

Both machines are nominally similar, same 8 core AMD Ryzen CPU, same RAM type & quantity, same AMD B550 chipset. Different mITX motherboard manufacturers MSI v Gigabyte, both on latest firmware. NVME OS drive plus SATA SSD's for /home

Only significant difference is the Debian box has 4 port Intel X710 NIC, FreeBSD has a 2 port Intel X710 NIC plus a 4 port Intel I350 NIC. Testing the FreeBSD box with & without the I350 NIC only delivers around 0.5W difference in standby/suspend

A very different 16 core AMD ZEN4 chip/motherboard combination running Debian 13 delivers similar low power consumption to the Debian 12 box in suspend

(only positive is the FreeBSD suspends and resumes to a usable state more quickly than the Debian boxes)

Spica · Monday at 10:25 AM

Debian 12

Direct off the power meter..

Maturin · Monday at 10:29 AM

I don't want to contradict your FreeBSD consumes more than your debian system, but
especially when it comes to power consumption you always need to look very closely and carefully what exactly you measure and compare.
To get a trustworthy and reliable value for the overall powerconsumption of a machine I recommend to do the measurement with a good power meter placed between the socket and the main power cord.
I know there are all kinds of such tools implemented in any computers today, like those you have in cars. It's not just only fantastic, what those all show (in nice, fancy graphs, with detailed statistics, and shit), but above all that almost all people blindely believe that, "because a computer presents them values with one digit after the decimal point."
As long as I don't have a reference measurement with an independant meter I trust I don't give a shit on what any computer tells me. It does not have to be wrong. Many measure actually correct. But I have too much experience in how those 'measurements' are done, what BS they can produce, and what's shameless presented to users, to trust any of those blindely.

Spica said:
Both machines are nominally similar, same 8 core AMD Ryzen CPU, same RAM type & quantity, same

Are the PSU also the same? Those not only can make a huge difference, but they are most responsible for the overall powerconsumption, especially in suspend mode.

To get a real comparable value, you better run an exact copy of both system on each machine, and measure with a power meter. Yes, that's effort. But this is the only way to get measurment values you can actually trust and really compare. Not having access to your machines and all the details I bet if you do so you will find a significant difference - between the measurement results you got the one way and the other.

Edit:
I saw the power meter post after I placed my post. However: This is an open forum, so the value should be to any reader, and don't forget to compare the PSUs.

Spica · Monday at 10:58 AM

Maturin said:
To get a trustworthy and reliable value for the overall powerconsumption of a machine I recommend to do the measurement with a good power meter placed between the socket and the main power cord.

The real time measurements are taken directly off the power cord to each machine using Shelly 1PM (4th Gen) inline power meters. These are not calibrated devices but have been tested against single phase ABB & Gavazzi
energy analyzers which do conform to EN IEC 62053-21, EN50470-3 calibrated and legally approved for domestic/industrial power measurement.

Measurement is within 1/10 of a Watt which is close enough to enable a reliable comparison.

The history graphs in post #1 are showing 5 minute average with smoothing applied, if you zoom in you can see the power consumption bumping up & down in realtime..

Maturin said:
Are the PSU also the same?

Yes, both Corsair SF750 Platinum rated running at a nominal 230V.

Maturin said:
To get a real comparable value, you better run an exact copy of both system on each machine

System specs are close enough, as is the software workload (not really relevant in standby)

Maturin · Monday at 11:04 AM

I was just checking that we talk actual comparable values here.

Spica · Monday at 12:39 PM

Intel X710 NIC power spec .. (SR-IOV not enabled)

Code:

> doas pciconf -lcv ixl0
Password:
ixl0@pci0:16:0:0:    class=0x020000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1572 subvendor=0x1dcf subdevice=0x0304
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller X710 for 10GbE SFP+'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 129 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x1000]
    cap 10[a0] = PCI-Express 2 endpoint max data 512(2048) FLR RO
                 max read 512
                 link x8(x8) speed 8.0(8.0) ASPM L1(L1)
    ecap 0001[100] = AER 2 0 fatal 1 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 801622ffff85b798
    ecap 000e[150] = ARI 1
    ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
                     0 VFs configured out of 64 supported
                     First VF RID Offset 0x0110, VF RID Stride 0x0001
                     VF Device ID 0x154c
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
    ecap 0017[1a0] = TPH Requester 1
    ecap 000d[1b0] = ACS 1 Source Validation unavailable, Translation Blocking unavailable
                     P2P Req Redirect unavailable, P2P Cmpl Redirect unavailable
                     P2P Upstream Forwarding unavailable, P2P Egress Control unavailable
                     P2P Direct Translated unavailable, Enhanced Capability unavailable
    ecap 0019[1d0] = PCIe Sec 1 lane errors 0

Supports Active State Power Management (ASPM) Level 1

Intel I350-T4 (SR-IOV not enabled)

Code:

> doas pciconf -lcv igb0
Password:
igb0@pci0:33:0:0:    class=0x020000 rev=0x01 hdr=0x00 vendor=0x8086 device=0x1521 subvendor=0x8086 subdevice=0x0001
    vendor     = 'Intel Corporation'
    device     = 'I350 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 10 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR RO NS
                 max read 512
                 link x4(x4) speed 5.0(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[140] = Serial 1 98b785ffff221814
    ecap 000e[150] = ARI 1
    ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
                     0 VFs configured out of 8 supported
                     First VF RID Offset 0x0180, VF RID Stride 0x0004
                     VF Device ID 0x1520
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
    ecap 0017[1a0] = TPH Requester 1
    ecap 0018[1c0] = LTR 1
    ecap 000d[1d0] = ACS 1 Source Validation unavailable, Translation Blocking unavailable
                     P2P Req Redirect unavailable, P2P Cmpl Redirect unavailable
                     P2P Upstream Forwarding unavailable, P2P Egress Control unavailable
                     P2P Direct Translated unavailable, Enhanced Capability unavailable

Doesn't support ASPM, although ASPM is switched on via Motherboard BIOS firmware.

Can't seem to change the state (of the I350 NIC) with any control knobs either... Although it's not clear to me that this would have any impact when the NIC is not "Active", but maybe not supporting L1 ASPM keeps the card active continuously (Intel specs says the card TDP is 4W, which doesn't fully cover the difference in suspended mode.

Code:

> doas sysctl hw.pci.enable_aspm
Password:

hw.pci.enable_aspm: 1

Spica · Tuesday at 11:09 AM

Some more background information from the handbook

Chapter 14. Configuration, Services, Logging and Power Management

This chapter explains much of the FreeBSD configuration files, how to enable or disable a service, how to configure the logging system and the power management area.

docs.freebsd.org

Code:

> sysctl hw.acpi.supported_sleep_state
hw.acpi.supported_sleep_state: S3 S4 S5

Reports correctly that S3 suspend to RAM is supported, which works reliably from either xfce4 > suspend or via the console with

Code:

> cat /etc/rc.conf | grep powerd
powerd_enable="YES"

powerd is running ..

Code:

> doas service powerd start           
Password:

powerd already running?  (pid=2710)

Code:

# acpiconf -s 3

Probably unrelated to high power consumption in sleep mode but I'm struggling to understand how to enable CPU frequency scaling on this AMD CPU and not finding much information other than related to branded OEM laptops

Code:

> doas sysctl dev.cpufreq.0.freq_driver dev.cpu.0.freq_levels dev.cpu.0.freq                   
Password:

dev.cpufreq.0.freq_driver: hwpstate0
dev.cpu.0.freq_levels: 3800/5130 1700/1615 1400/1277
dev.cpu.0.freq: 3800

> sysctl dev.cpu.0
dev.cpu.0.temperature: 31.1C
dev.cpu.0.cx_method: C1/mwait C2/io C3/io
dev.cpu.0.cx_usage_counters: 110348797 0 0
dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 349us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/1 C2/2/18 C3/3/350
dev.cpu.0.freq_levels: 3800/5130 1700/1615 1400/1277
dev.cpu.0.freq: 3800
dev.cpu.0.%iommu:
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=ACPI0007 _UID=0 _CID=none
dev.cpu.0.%location: handle=\_SB_.PLTF.C000
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU

The machine sits stubbornly at 3800MHz all the time ..

Is this the expected behavior for a desktop machine, high power consumption in sleep and no CPU scaling with AMD CPU's?