FreeBSD 50% higher power consumption than Debian

The Debian and FreeBSD installations are without GUI.
Debian's kernel by default comes with and loads in the DRM/KMS i915 kernel module.

With FreeBSD you should do that too or it will be using scfb or vesa which has very basic power management. This is important, even if you aren't planning on running Xorg. It affects the framebuffer terminal too.

I.e in theory, Windows has the best power management for consumer amd64 hardware but if you also run it with the "Basic Video Adapter", it is pretty terrible too.

Otherwise, I might suspect it is because FreeBSD has fairly early Intel p-state support. It might be disabled by default due to this bug.
 
That can't work on weberjn's FUTRO S720 as it has no battery, so devd cannot run /etc/rc.d/power_profile on notifies from ACPI ACAD, so sysctl hw.acpi.cpu.cx_lowest never changes, so nor sysctl dev.cpu.{0,1}.cx_lowest - however these may be set in /etc/sysctl.conf.
what has a battery to do with the C-states? Those entries set the "hw.acpi.cpu.cx_lowest" as well as all "dev.cpu.N.cx_lowest" to the lowest state supported by the OS (C8):
Code:
# sysctl -a | grep cx_lowest
hw.acpi.cpu.cx_lowest: C8
dev.cpu.11.cx_lowest: C8
dev.cpu.9.cx_lowest: C8
dev.cpu.7.cx_lowest: C8
dev.cpu.5.cx_lowest: C8
dev.cpu.3.cx_lowest: C8
dev.cpu.1.cx_lowest: C8
dev.cpu.10.cx_lowest: C8
dev.cpu.8.cx_lowest: C8
dev.cpu.6.cx_lowest: C8
dev.cpu.4.cx_lowest: C8
dev.cpu.2.cx_lowest: C8
dev.cpu.0.cx_lowest: C8
# sysctl -a | grep cx_supp
dev.cpu.11.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.9.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.7.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.5.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.3.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.1.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.10.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.8.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.6.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.4.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.2.cx_supported: C1/1/1 C2/2/253 C3/3/1048
dev.cpu.0.cx_supported: C1/1/1 C2/2/253 C3/3/1048
# grep cx_low /etc/rc.conf
performance_cx_lowest="Cmax"
economy_cx_lowest="Cmax"
This is taken from my work desktop I'm currently sitting at. (no battery)

from rc.conf(5):
Code:
    performance_cx_lowest
                 (str) CPU idle state to use while on AC power.  The string
                 “LOW” indicates that acpi(4) should use the lowest power
                 state available while “HIGH” indicates that the lowest
                 latency state (less power savings) should be used.

[...]

     economy_cx_lowest
                 (str) CPU idle state to use when off AC power.  The string
                 “LOW” indicates that acpi(4) should use the lowest power
                 state available while “HIGH” indicates that the lowest
                 latency state (less power savings) should be used.

So on normal systems "performance_cx_lowest" should be sufficient and "economy_cx_lowest" is only necessary for systems that actually *can* be off AC power (i.e. have a battery).
I have both lines in my rc.conf template, so it ends up on all systems I set up and does no harm.


Which systems? I can't see how that could be possible, when changing CPU speed doesn't affect C-states?
The cx_usage statistics were slightly shifted towards the higher (i.e. numerical lower) states on average when I ran powerdxx. However; power readings from IPMI didn't show any difference, so the actual impact might be lost in normal fluctuations. But as reducing clock rates has zero additional impact on power consumption with enabled (and working) C-states, yet sometims significant impact on performance, especially because powerdxx often fails to set the clock rates to turbo-boost frequency, I don't use it anywhere.
This was tested on 4 (2 pairs of identical) servers with dual Xeon E5 v4 (Broadwell), where I set Cmax and then enabled powerd(xx) only on one of each pair. Power readings were gathered via zabbix by running a script that basically executes ipmitool dcmi power reading.


Even C-state C8 doesn't 'shut down a core', and reducing clock speed according to load (esp. no load) makes a large difference in power use on any processor I've seen.
C2 - Stop Clock: Core and bus clocks are off. The processor maintains all software-visible state, but can take longer to wake up.
C3 - Deep Sleep: Clock generator is off. The processor does not need to keep its cache coherent, but maintains other states. Some processors have variations of the C3 state (Deep Sleep, Deeper Sleep) that differ by how long it takes to wake the processor.
C4 - Deeper Sleep: Reduced VCC
Beginning with C2 some or all (C3) clock generators are off - this means that changing the clock rates doesn't make any sense or difference any more. At C4 and higher the VCC is reduced, at C6 this explicitly includes 0V, which is basically "shutting down" the core.

Well I've only used older Intel CPUs in recent decades, which ones do you refer to?
I have various systems at hand, starting from haswell/broadwell, skylake, some jasper lake up to Tiger and Alder Lake. (no AMD though - last one I used was an Athlon 64)
On *all* those systems (servers, appliances/routers, desktops, laptop...) I use those cx_lowest settings in /etc/rc.conf and they successfully set the cx_lowest sysctls to "C8".
 
Let me remind that the question is not if FreeBSD can do C states, but why Debian 12 needs less power than FreeBSD 13.2 with default settings on the same hardware.

And the most promising hint seems to me the one of kpedersen:

> Debian's kernel by default comes with and loads in the DRM/KMS i915 kernel module.
 
Let me remind that the question is not if FreeBSD can do C states, but why Debian 12 needs less power than FreeBSD 13.2 with default settings on the same hardware.

It's generally a bit like comparing apples to oranges - take another linux distribution and they will have other defaults set and drivers loaded (possibly with other default settings/compile options).
Generally FreeBSD is often geared towards maximum compatibility and conservative settings. Power saving options have often caused problems with broken or incomplete implementations (especially on desktop and laptop hardware), so IMHO it's a good thing those things are off or very conservatively set.

I'd also suspect that the difference might be because debian loads GPU drivers by default, which may enable some power savings there. So to eliminate that factor when comparing, either both OS have to have loaded drivers for the same set of hardware, or the GPU has to be completely disabled at the BIOS (if possible).
 
Looks like it was fixed and Merged From Current back in February?

That's true. Though I noticed subsequent to the merge that there were sporadic reports of it still "wedging". (I temporarily had a newer X1 Carbon at work that exhibited this problem, so for a while I was keen on following reports relating to this). I thought they just left it off for the next release cycle.
 
Let me remind that the question is not if FreeBSD can do C states, but why Debian 12 needs less power than FreeBSD 13.2 with default settings on the same hardware.
It is just something that some distros do. For example, when you install wpa_supplicant, it automatically enables it. FreeBSD doesn't do that; you end up with a more vanilla and deterministic experience.

In many ways, I find it weird that Debian "half" enables some power management stuff, but not all of them. For example, you will still want to tweak the USB power for unused devices. You will likely also want to change cpufreq governor to one more appropriate for your CPU.

When it comes down to all or nothing, I actually find FreeBSD's approach much cleaner. That said, when I do install Linux (usually Debian), I tend to do a completely manual install via debootstrap. The random "fiddling" that the Debian installer does is too messy for me. And then, in this case, much of the power management is unconfigured. I almost see the Debian installer as quick and dirty for new users with a wide selection of consumer hardware that it gets 80% right.
 
Looks like it was fixed and Merged From Current back in February?

Red herring, methinks. This thread is about an AMD CPU on GX-217GA SoC, so I doubt an Intel fix is relevant.

Personally I'm finding this thread a bit odd; results of one system with largely unknown configuration for one usage case - minimum power draw doing nothing at all - without tuning for any real usage case can't be extrapolated for other cases.

Later hoping to elicit deeper data about setup, like power states on all those USB ports and unused pci devices etc.
 
what has a battery to do with the C-states? Those entries set the "hw.acpi.cpu.cx_lowest" as well as all "dev.cpu.N.cx_lowest" to the lowest state supported by the OS (C8):

Quite so. I'd forgotten that /etc/rc runs /etc/rc.d/* (plus local) scripts on startup, including power_profile, so only performance_ settings are relevant without battery.

I'll have to get back to you later about C-states, powerd and power consumption after some more testing on my T430s.
 
You could do the following: run the CPU at 100% on both systems. If the power consumption is still with the same delta, it's not the CPU doing this. I would say the GPU is doing this here.
 
my 2 cents. FreeBSD motto is "Power to Serve", not "Power to Spare". It is a OS primarily dedicated to servers, it arrives to you configured for a server. I could take power measurement on a few of my machines as well, but I wish you to focus on another point as well. FreeBSD takes 50% time LESS to run a basic command via ssh than Debian on a BeagleBone Black. Both system are not tweaked and run as they are. They are a few km away, under the same VPN, and similar latency. Experiment below:

There is for sure a configuration that makes by default Debian less sleepy and more power consuming => faster. For my typical use cases i prefer the FreeBSD default and this is what i expect from it.

Code:
#. "pulce1" is a BBB with Debian 11 (the latest official for that box)
p@deb4:~/download$ time ssh p@wg1-pulce1 hostname
Debian GNU/Linux 11 \n \l

Known users have access through ssh pulic key.
pulce1

real    0m2.922s
user    0m0.034s
sys     0m0.000s

# . "pulce2" is a FreeBSD 12.3, fresh install (one week aog)
p@deb4:~/download$ time ssh p@wg1-pulce2 hostname
pulce2

real    0m1.292s
user    0m0.146s
sys     0m0.004s
 
I've repeated the exercise with a Thinkpad E130, battery removed.
Intel(R) Core(TM) i3-3217U CPU @ 1.80GHz

34% higher power consumption.

Debian: 11,7 W idle
FreeBSD: 15,7 W idle

I have a similar era T430s, Core i5-3320M @2.6GHz, 8G RAM. Initially everything was enabled ('power to serve') so it ran about 15W on battery.

After basic tuning (USB and i915 powersave modes; fingerprint reader, bluetooth, webcam and unused PCI off; no X, 55% backlight, wifi on) it idles at 8.1W on battery.

That doesn't prove that Debian runs 44% higher power consumption on a similar system, but does show the futility of comparing untuned systems having different tuning paradigms or priorities.

sysctl dev.cpu.{0,1}.cx_usage
dev.cpu.0.cx_usage: 1.03% 98.96% last 3712us
dev.cpu.1.cx_usage: 2.99% 97.00% last 65674us

Similarly ~99% C2 for cpu.0-3 on AC, but on battery the T430s idles at ~99% in C3.
 
After basic tuning (USB and i915 powersave modes; fingerprint reader, bluetooth, webcam and unused PCI off; no X, 55% backlight, wifi on) it idles at 8.1W on battery.

Could you please quote the commands and settings you used?
 
Could you please quote the commands and settings you used?

Sure. Just basic tuning, look for tutorials by mav@ and Vermaden among others if you want to get serious. [edit: these are on 12.4]

Code:
# 28/8/23 t430s mostly just power related settings
smithi@t430s:~ % egrep -v '^$|^#' /boot/loader.conf
acpi_ibm_load="YES"
acpi_video_load="YES"   # for brightness control
coretemp_load="YES"
boot_verbose="YES"
verbose_loading="YES"   # log loaded modules
cpuctl_load="YES"           # for powermon
acpi_call_load="YES"
hw.psm.synaptics_support="1"
kern.vt.fb.modes.LVDS-1="800x600"
compat.linuxkpi.i915_enable_dc=2        # tunable: i915kms load by kld_list
compat.linuxkpi.i915_enable_fbc=1       # sysctl, ok here? if not, rc.local

smithi@t430s:~ % egrep -v '^$|^#' /etc/rc.conf
powerd_enable="YES"     # dflts: -a hadp -i50 -r75 -M2601 -m1200 -p250 -N
powerd_flags="-a adp -b adp"
performance_cx_lowest="Cmax"
economy_cx_lowest="Cmax"
kld_list="/boot/modules/i915kms.ko fusefs cuse"         # +exfat, webcamd
webcamd_enable="YES"
webcamd_0_flags="-N Ricoh-Company-Ltd--Integrated-Camera"       # 7/12/22

smithi@t430s:~ % egrep -v '^$|^#' /etc/sysctl.conf
dev.acpi_ibm.0.handlerevents="0x10 0x11"        # replace these also ..
hw.acpi.cpu.cx_lowest=Cmax
hw.acpi.verbose=1
debug.acpi.resume_beep=1        # not working beep in X?
hw.pci.do_power_nodriver=3
vfs.usermount=1
net.inet.ip.fw.autoinc_step=20  # was 50, dflt 100
hw.acpi.video.lcd0.fullpower=65
hw.acpi.video.lcd0.economy=55
dev.acpi_ibm.0.bluetooth=0

smithi@t430s:~ % egrep -v '^$|^#' /etc/rc.local
/usr/sbin/usbconfig -d 1.3 power_off    # fingerprint scanner nom. 100mA
/usr/sbin/usbconfig -d 1.4 power_off    # bluetooth - saves ~500mW!
/usr/sbin/usbconfig -d 1.5 power_save   # camera nom. 200mA
ifconfig wlan0 powersave
sysctl compat.linuxkpi.i915_enable_fbc=1

/etc/rc.resume: otherwise turns back on; fix when using. NB now ugen1.3
usbconfig -d `usbconfig | grep Camera | awk '{ print $1 }'` power_off
 
my non-embedded experience, recently installed freebsd on razer blade laptop. I gave up its windows because of hot cpu. and then installed linux, and finally freebsd. it worked but still hot (.. hm.. less than by windows)
 
I tried the settings of smithi, but I see no change in power consumption on the E130.
Then I tried graphics/drm-kmod kld_list="i915kms"
and the settings from this post, still no change.
 
Good to see that the Pi is running with less power.

I suspect it is because less hardware is initialized due to lacking support in FreeBSD (wifi?).
 
and yet another one: Lenovo ThinkCentre M710Q, i3-6100T

Debian 12: 6 Watt
FreeBSD 13.2: 8.2 Watt

This one has hwpstate_intel0: <Intel Speed Shift>
so powerd is useless, I guess.
 
Especially for a professional server OS low power consumption is an important competitive advantage, at least here in Europe where energy is expensive.
Energy consumption is for a server farm a cost factor as gas consumption is for a fleet of trucks.
 
Especially for a professional server OS low power consumption is an important competitive advantage, at least here in Europe where energy is expensive.
Have you measured on server hardware?

I find on home stuff and laptops, Linux wins ~70% of the time. FreeBSD wins the others. On old hardware, I find FreeBSD almost always wins.

For recent server stuff (HP, IBM 2U mainly), I find they are weirdly equal. For aging servers (probably 90% of the world), FreeBSD tends to do better.
 
I'd hope somebody has and can measure.
Annoyingly I had some fairly good notes on a range of hardware but I threw it away whilst moving office.

I will try to get some readings on recent hardware again.

That said, in the UK, running a Pi 3 (around 2-3 watts) 24/7, costs me under £5 a year. So in many ways it isn't worth the effort if there is only ~3 watts difference between FreeBSD and Linux. Even though for laptops it can make the difference between feeling warm.

With Linux you would end up spending that amount of cost in time trying to diagnose bad decisions :D
 
Back
Top