Power management on FreeBSD

[larshenrikoern]

Hi

I have recently looked a lot into FreeBSD power management. The reason was that my processor (i7-8700) became quite hot when compiling ports. So the first thing I did was to renew the CPU paste on the CPU. And did help quite a bit. Right now I am frustrated because it is really try and error to get things going. One of the big problems is that the documentation is quite old, and that there are so many small tools that need to work together. I have tried different combinations of powerd, C-states, Intel Speed shift, Bios settings and powerdxx.
And Intel Speed shift seems to cause a lot of trouble (and more heat). Right now I am using default bios settings, and powerdxx. Below are my loader.conf, rc.conf and sysctl.conf.

Coments and sugestions most welcome

 

[SirDice]

The reason was that my processor (i7-8700) became quite hot when compiling ports.

Compiling is a pretty CPU intensive job, if/when the CPU has been chugging along at 99% usage the CPU will heat up significantly. That's why CPUs have big fat coolers on them nowadays.

So the first thing I did was to renew the CPU paste on the CPU.

Good contact between the CPU and the cooler is important. But having good airflow in the case is too. That cooling block has to disperse that heat to the environment and it can't do that efficiently if the environment is stagnant and traps the heat.

 

[larshenrikoern]

Compiling is a pretty CPU intensive job, if/when the CPU has been chugging along at 99% usage the CPU will heat up significantly. That's why CPUs have big fat coolers on them nowadays.


Good contact between the CPU and the cooler is important. But having good airflow in the case is too. That cooling block has to disperse that heat to the environment and it can't do that efficiently if the environment is stagnant and traps the heat.

My temperature was before replacing the cpu paste around 95 degree celcius and A very busy Fan. After replacing it is at full load around 85 degree celsius and a less noisy fan. So it is now running more acceptable. But in the process a lot of questions about the state of FreeBSD and power came up. And for instance https://wiki.freebsd.org/TuningPowerConsumption did create just as many problems as it did solve.

 

[T-Daemon]

My temperature was before replacing the cpu paste around 95 degree celcius and A very busy Fan. After replacing it is at full load around 85 degree celsius

The Intel® Core™ i7-8700 Processor's Junction Temperature is 100°C, if necessary you can lower the temperature by assigning fewer cores with cpuset(1) to the make(1) target.

 

[SirDice]

Power management throttles the CPU to a lower frequency. A CPU running at a lower frequency is slower and consequently uses less energy and produces less heat. This is fine for background tasks that don't use much CPU to begin with. When compiling (or rendering, or some other CPU intensive job) you typically want the CPU to run as fast as possible, even getting it to use their 'boost' clock speeds. This inevitably causes the CPU to use more energy and produce more heat.

My temperature was before replacing the cpu paste around 95 degree celcius and A very busy Fan. After replacing it is at full load around 85 degree celsius and a less noisy fan.

This really tells me you have the wrong sized cooler[*] or there's simply not enough airflow in the case. Why do you think overclockers like Jayztwocents and Gamers Nexus always use those big water cooled blocks? Heck, they've even experimented with liquid nitrogen to cool the CPU.

[*] Is it the stock cooler that came with the CPU? Replace it with a proper aftermarket cooler. Those stock coolers are fine for every day use but quickly fall apart when you're constantly pushing the CPU to the max.

 

[larshenrikoern]

Power management throttles the CPU to a lower frequency. A CPU running at a lower frequency is slower and consequently uses less energy and produces less heat. This is fine for background tasks that don't use much CPU to begin with. When compiling (or rendering, or some other CPU intensive job) you typically want the CPU to run as fast as possible, even getting it to use their 'boost' clock speeds. This inevitably causes the CPU to use more energy and produce more heat.


This really tells me you have the wrong sized cooler[*] or there's simply not enough airflow in the case. Why do you think overclockers like Jayztwocents and Gamers Nexus always use those big water cooled blocks? Heck, they've even experimented with liquid nitrogen to cool the CPU.

[*] Is it the stock cooler that came with the CPU? Replace it with a proper aftermarket cooler. Those stock coolers are fine for every day use but quickly fall apart when you're constantly pushing the CPU to the max.

The problem is that it is a HP prebuilt unit. And everything in it is custom made for HP. So the coolermount is non standard. The power supply is also custom. So I cant change the cooler, except for building a new PC . But around 85 degree celsius is also acceptable

 

[larshenrikoern]

Back to my frustration. What combo of settings gives the best possible cooling (at idle). Currently powerdxx and dev.cpu.X.cx_lowest=C2 in sysctl.conf is OK. But if it can be done better I would like to try that out.

 

[SirDice]

And everything in it is custom made for HP. So the coolermount is non standard.

CPU socket is standard (LGA1151 in this case), so the mounting holes for the cooler are too. You may need to remove and replace the base plate of the cooler though. Because that base plate is specific to the cooler.

 

[larshenrikoern]

CPU socket is standard (LGA1151 in this case), so the mounting holes for the cooler are too. You may need to remove and replace the base plate of the cooler though. Because that base plate is specific to the cooler.

Right now the cooling under load is OK. My question is what combo of tools and settings on the FreeBSD side gives the best ballance between performance and energy consumption. ?? When idle the temperature is 35 degree celcius. And I think that is just quite normal and expected

 

[SirDice]

When idle the temperature is 35 degree celcius. And I think that is just quite normal and expected

Depends a bit on the ambient temperature in the room but 35C is good. All my CPUs are typically idling at that temperature too. During the heat wave last week it got a bit higher though. While pushing my new desktop (i7-11700K) it will climb up to around 55C, but I have a rather oversized AIO water cooler on it.

If replacing the cooler isn't much of an option, is there a place to add a (additional) case fan? More airflow through the case would mean the cooler can dissipate its heat more, which should result in lower temperatures overall.

 

[larshenrikoern]

I have replaced the paste once more and tries to compile firefox. The temperature is now hover around 80 degrees Celcius and an even slower fan. So a bit better (tried another applying method). And nothing to worry about as I see it. Today it is a bit hot so ambient temperature in the room around 26 degrees.

What is left is a better explanation of the setup for a workstation. And the pros and cons of the different tool and their combinations for a good compromise between power and speed.

 

[larshenrikoern]

Power management throttles the CPU to a lower frequency. A CPU running at a lower frequency is slower and consequently uses less energy and produces less heat. This is fine for background tasks that don't use much CPU to begin with. When compiling (or rendering, or some other CPU intensive job) you typically want the CPU to run as fast as possible, even getting it to use their 'boost' clock speeds. This inevitably causes the CPU to use more energy and produce more heat.


This really tells me you have the wrong sized cooler[*] or there's simply not enough airflow in the case. Why do you think overclockers like Jayztwocents and Gamers Nexus always use those big water cooled blocks? Heck, they've even experimented with liquid nitrogen to cool the CPU.

[*] Is it the stock cooler that came with the CPU? Replace it with a proper aftermarket cooler. Those stock coolers are fine for every day use but quickly fall apart when you're constantly pushing the CPU to the max.

Following your advice I have now removed all power management. Bios is set to default settings (regarding power). And the only things I have left is machdep.hwpstate_pkg_ctrl=0 in loader.conf, and
dev.cpu.0.cx_lowest=C1
dev.cpu.1.cx_lowest=C2
dev.cpu.2.cx_lowest=C2
dev.cpu.3.cx_lowest=C2
dev.cpu.4.cx_lowest=C2
dev.cpu.5.cx_lowest=C2
dev.cpu.6.cx_lowest=C2
dev.cpu.7.cx_lowest=C2
dev.cpu.8.cx_lowest=C2
dev.cpu.9.cx_lowest=C2
dev.cpu.10.cx_lowest=C2
dev.cpu.11.cx_lowest=C2

in sysctl.conf (I have 12 cores)

The temperatures are the same as before with powerdxx enabled (around 80 during compile). But the performance seems a bit faster. I have tried for several hours, with almost every possible combination of settings (observing temperatures during compile of Firefox and idle) with bios power settings, powerd powerdxx, C states (indivdual and performance_cx_lowest="Cmax" economy_cx_lowest="Cmax" in rc.conf)

My recommendation for a desktop is after this the above

 

[smithi]

Following your advice I have now removed all power management.

I missed that? A bit drastic, based on a few hours testing, I'd have thought. It takes time and good logging to strike the right balance, every system is different, so it's hard to give general advice. However:

Bios is set to default settings (regarding power).

FreeBSD historically ignores BIOS settings regarding power.

And the only things I have left is machdep.hwpstate_pkg_ctrl=0 in loader.conf, and
dev.cpu.0.cx_lowest=C1
dev.cpu.1.cx_lowest=C2
dev.cpu.2.cx_lowest=C2
dev.cpu.3.cx_lowest=C2
dev.cpu.4.cx_lowest=C2
dev.cpu.5.cx_lowest=C2
dev.cpu.6.cx_lowest=C2
dev.cpu.7.cx_lowest=C2
dev.cpu.8.cx_lowest=C2
dev.cpu.9.cx_lowest=C2
dev.cpu.10.cx_lowest=C2
dev.cpu.11.cx_lowest=C2

in sysctl.conf (I have 12 cores)

I can't fathom having cpu.0 set differently? Does thst actually take, when you display the sysctls later? In any case sysctl hw.acpi.cpu.cx_lowest=Cmax ( ie C8) is usually worthwhile; no impediment when you need performance, cooling benefits when nearer idle, and that setting propagates to each cpu the same, ie no need for above.

The temperatures are the same as before with powerdxx enabled (around 80 during compile). But the performance seems a bit faster. I have tried for several hours, with almost every possible combination of settings (observing temperatures during compile of Firefox and idle) with bios power settings, powerd powerdxx, C states (indivdual and performance_cx_lowest="Cmax" economy_cx_lowest="Cmax" in rc.conf)

The latter cx_lowest settings are only used when switching between AC power and battery, ie for laptops (but won't hurt).

In powerd you can set max CPU speed with -M, so if your
sysctl dev.cpu.0.freq_levels starts with (e.g. here) 2601/35000 2600/35000 ... then -M 2600 turns off Turbo speed. Likely not what you want on your beast, but handy if e.g. make world or big ports is dying from overheating. {thanks, Trev} Does powerdxx offer that setting also?

My recommendation for a desktop is after this the above

Sounds like you've made a good start on your big beastie!

 

[larshenrikoern]

I missed that? A bit drastic, based on a few hours testing, I'd have thought. It takes time and good logging to strike the right balance, every system is different, so it's hard to give general advice. However:


FreeBSD historically ignores BIOS settings regarding power.



I can't fathom having cpu.0 set differently? Does thst actually take, when you display the sysctls later? In any case sysctl hw.acpi.cpu.cx_lowest=Cmax ( ie C8) is usually worthwhile; no impediment when you need performance, cooling benefits when nearer idle, and that setting propagates to each cpu the same, ie no need for above.



The latter cx_lowest settings are only used when switching between AC power and battery, ie for laptops (but won't hurt).

In powerd you can set max CPU speed with -M, so if your
sysctl dev.cpu.0.freq_levels starts with (e.g. here) 2601/35000 2600/35000 ... then -M 2600 turns off Turbo speed. Likely not what you want on your beast, but handy if e.g. make world or big ports is dying from overheating. {thanks, Trev} Does powerdxx offer that setting also?



Sounds like you've made a good start on your big beastie!

sysctl does indeed honor having cpu.0 set differently. I had this from Vermeer. And it does make response times a tiny bit shorter that one cpu is ready at all times.

 

[larshenrikoern]

I have now replaced the cpu fan on my motherboard. But buuuh, Difficult. The mouning was not standard screwes on my system, the case to small. Looked and did find some screws that cold get old and new to work together. And mr Gaffa made the to small case closed again . The temperatures now maxing out around 70-75 degree celcius. An improvement from 90

 

[smithi]

sysctl does indeed honor having cpu.0 set differently. I had this from Vermeer. And it does make response times a tiny bit shorter that one cpu is ready at all times.

Well, I guess, if saving at most a few hundred nanoseconds gets the job done sooner, but in human terms it is tiny <&^}=

But I have to admit to being 4 years out of touch with latest C and P-state stuff, so I should shutup till I can dig a bit.

 

[Styrsven]

I found that powerdxx can control according to temperature.
I have powerdxx_flags="-n adaptive -a hiadaptive -b adaptive -H 75C:80C" in my rc.conf and the temp never rises above 75.5 when poudriere does its thing.
My processor is a Ryzen, but I don't think that matters.

 

[flipper_88]

There are quite a few off lease workstations on eBay that are reasonably priced, however be careful of which seller you by from because some of them are not as honest as they look. I'd know because I recently bought Aa Lenovo ThinkPad L450 from a seller on eBay and am now waiting for the hard-drive mounting bracket to come from amazon as it was not included with the laptop from the eBay seller.

 

[larshenrikoern]

There are quite a few off lease workstations on eBay that are reasonably priced, however be careful of which seller you by from because some of them are not as honest as they look. I'd know because I recently bought Aa Lenovo ThinkPad L450 from a seller on eBay and am now waiting for the hard-drive mounting bracket to come from amazon as it was not included with the laptop from the eBay seller.

Hi

Are you answering to the wrong thread ??. This thread is about power management on FreeBSD

 

[kpedersen]

Well, I guess, if saving at most a few hundred nanoseconds gets the job done sooner

My reasoning against that (and instead putting everything to Cmax/C8) is that the lowest my CPU can be scaled to is 1200 anyway. If things (mainly GUIs) feel sluggish with that, I replace the software with something better.

 

[kpedersen]

larshenrikoern, It would be interesting to compare the energy usage with Linux (and even Windows). Does it run hot with them too?

It might be a big ask but do you perhaps have a plug like this that can measure the electricity usage between them?

My own personal experience (with a HP Z420) is that FreeBSD uses ~5% more energy than Linux. Since Windows is such a mess, I can't get a consistent reading haha. It can reach lower than both (presumably due to focused vendor drivers) but it is constantly faffing like a child with ADHD so keeps waking itself up from lower Cstates.

 

[gomoto]

I found that powerdxx can control according to temperature.
I have powerdxx_flags="-n adaptive -a hiadaptive -b adaptive -H 75C:80C" in my rc.conf and the temp never rises above 75.5 when poudriere does its thing.
My processor is a Ryzen, but I don't think that matters.

Thank you a lot. I used your settings on my Ryzen 3600x system to stop my Chromium crashes.

FreeBSD and Chromium are rock stable on my Celeron J4125 System. But I could not pinpoint the browser crashes on my Ryzen 3600x. Thanks for sharing your experiences with your Ryzen CPU.