Tweaking FreeBSD for Desktop Use - kern.sched.preempt_thresh

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

When looking at how to tweak FreeBSD for desktop use, the most common tweak is this:

Code:
sysctl kern.sched.preempt_thresh=224
Default: 80

This, supposedly, does something with X Windows, allowing it to run more smoothly when CPU use is very high... or something like that...

My question is: What does it mean, and what does it do?

Also, I can't set it in /boot/loader.conf... how do I set it?

Also, any things you'd recommend doing, before a custom kernel, to optimize FreeBSD for desktop use? Anything you did you want to share?

I know of tuning(7), but it's not exactly a stimulating read... : <
 
Last edited by a moderator:

tobik@

Daemon
Developer

Reaction score: 1,418
Messages: 1,909

Also, any things you'd recommend doing, before a custom kernel, to optimize FreeBSD for desktop use?
Use the defaults first to find out where your pain points with them really are. Then look for solutions for each one in a directed manner iff you have any problems.
 
OP
poorandunlucky

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

See sysctl.conf(5), and you would use something like
Code:
kern.sched.preempt_thresh=224
in /etc/sysctl.conf.

Ohh... I should've checked... : \ Thanks...

How come some get set in /boot/loader.conf, though? Why don't they all get set at the same place?

Use the defaults first to find out where your pain points with them really are. Then look for solutions for each one in a directed manner iff you have any problems.

My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive, and I think that's what kern.sched.preempt_thresh was supposed to do, but I'm not sure exactly what it means, nor what value to set it at...

I'm not even sure if it does help, or if it's just coincidental...
 
Last edited by a moderator:

tobik@

Daemon
Developer

Reaction score: 1,418
Messages: 1,909

My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive, and I think that's what kern.sched.preempt_thresh was supposed to do, but I'm not sure exactly what it means, nor what value to set it at...
I wish I could tell you but I don't know. However it might suffice if you just reduce the number of builders or make jobs that run in parallel. If you use Poudriere that's configurable in /usr/local/etc/poudriere.conf via PARALLEL_JOBS. It's a good idea anyway if you use the build machine for something other than building packages.
 

sidetone

Daemon

Reaction score: 832
Messages: 1,728

Ohh... I should've checked... : \ Thanks...
You didn't know where to look, but it became obvious in hindsight. I use man -k, with grep to look for commands: many are based on their filenames.
How come some get set in /boot/loader.conf, though? Why don't they all get set at the same place?
Loader is mostly for what's needed to start basic services, and needed before everything else, such as what's needed for single user mode. /boot is on its own partition, or the first parts of your drive, and that's needed first. If it is needed to boot everything else, it won't work in /etc/rc.conf. In the words from loader.conf(5), it's a bootstrap. Your window manager isn't a lower level service, so stuff relating to it can go into /etc/rc.conf. Some services that seem they would fit into loader.conf actually go into rc.conf, because they can work like that.

My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive, and I think that's what kern.sched.preempt_thresh was supposed to do, but I'm not sure exactly what it means, nor what value to set it at...

I'm not even sure if it does help, or if it's just coincidental...
IDK if this applies, but if you're using a transparent terminal, it will take up more resources. It can also be because of a heavyweight window manager. Long compile times are typical on FreeBSD, but it's at least more efficient than other operating systems. You can use tmpfs(5) (not md(4) with tmpmfs which is also a RAM filesystem, but inefficient for this purpose) in /etc/rc.conf and /etc/fstab, to use RAM for faster compiling. If that is not enough, perhaps you can upgrade your CPU.

I use this for mounting a RAM compiling directory
/etc/fstab
Code:
tmpfs           /usr/obj        tmpfs   rw,mode=1777    0 0 # for kernel & world
tmpfs           /tmp            tmpfs   rw,mode=1777    0 0 # need config below for ports
You can add /var/tmp to the above, or you can softlink it to /tmp. A swap file system will also be needed, if there is not enough RAM for port compiles.

In /etc/make.conf
Code:
WRKDIRPREFIX=/tmp/bport
or any chosen filename is needed for ports.
 
OP
poorandunlucky

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

You didn't know where to look, but it became obvious in hindsight. I use man -k, with grep to look for commands: many are based on their filenames.

Loader is mostly for what's needed to start basic services, and needed before everything else, such as what's needed for single user mode. /boot is on its own partition, or the first parts of your drive, and that's needed first. If it is needed to boot everything else, it won't work in /etc/rc.conf. In the words from loader.conf(5), it's a bootstrap. Your window manager isn't a lower level service, so stuff relating to it can go into /etc/rc.conf. Some services that seem they would fit into loader.conf actually go into rc.conf, because they can work like that.


IDK if this applies, but if you're using a transparent terminal, it will take up more resources. It can also be because of a heavyweight window manager. Long compile times are typical on FreeBSD, but it's at least more efficient than other operating systems. You can use tmpfs(5) (not md(4) with tmpmfs which is also a RAM filesystem, but inefficient for this purpose) in /etc/rc.conf and /etc/fstab, to use RAM for faster compiling. If that is not enough, perhaps you can upgrade your CPU.

I use this for mounting a RAM compiling directory
/etc/fstab
Code:
tmpfs           /usr/obj        tmpfs   rw,mode=1777    0 0 # for kernel & world
tmpfs           /tmp            tmpfs   rw,mode=1777    0 0 # need config below for ports
You can add /var/tmp to the above, or you can softlink it to /tmp. A swap file system will also be needed, if there is not enough RAM for port compiles.

In /etc/make.conf
Code:
WRKDIRPREFIX=/tmp/bport
or any chosen filename is needed for ports.

Poudriere does that, compile in RAM, I mean... The window manager being unresponsive is just what other people were saying that setting helped, I'm not sure if it really changes anything, I'd have to try twice the same thing, which I haven't...

I actually have another problem semi-related to compiling in RAM with Poudriere... it runs out of memory, and it creates a kernel panic, and everything just stops working...

I set Poudriere to use 6 builders, and 2 GB of RAM for the work dir + 1 GB of RAM for jails instead of 8 + unlimited to prevent that happening, but I'm not sure if that's optimal...

I'd rather just properly manage swap and resources than set limits for a particular program... like this is a big problem, IMO...

Here's the thread I'm talking about: https://forums.freebsd.org/threads/63384/
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,288
Messages: 4,099

/boot/loader.conf is for setting things before the kernel is loaded. This is mainly for things that can't be changed once the kernel is loaded.

/etc/sysctl.conf is for setting things after the kernel is loaded but before any users have logged in.

sysctl(8) is for settings things after the kernel is loaded and after a user has logged in. Mostly for changing/testing things on the fly before making the setting permanent by adding it to /etc/sysctl.conf.

The easy way to find out where the setting goes: try to change it form a shell using sysctl(8). If you get an error about the tunable being read-only, then it goes into loader.conf. If the change takes effect, then it goes in sysctl.conf.
 

rigoletto@

Daemon
Developer

Reaction score: 1,249
Messages: 2,291

Poudriere does that, compile in RAM, I mean...

Just in case, you need to set it in poudriere.conf

Code:
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
# data      - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all       - Run the entire build in memory, including builder jails.
# yes       - Enables tmpfs(5) for wrkdir and data
# no        - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS="wrkdir data"

I actually have another problem semi-related to compiling in RAM with Poudriere... it runs out of memory, and it creates a kernel panic, and everything just stops working...

You probably are running without SWAP, otherwise it will be pretty slow but not actually break. Or, you have a faulty memory.
 
OP
poorandunlucky

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

Just in case, you need to set it in poudriere.conf

Code:
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
# data      - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all       - Run the entire build in memory, including builder jails.
# yes       - Enables tmpfs(5) for wrkdir and data
# no        - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS="wrkdir data"



You probably are running without SWAP, otherwise it will be pretty slow but not actually break. Or, you have a faulty memory.

Code:
vm.swap_enabled: 1
vm.disable_swapspace_pageouts: 0
vm.swap_idle_enabled: 1
vm.stats.vm.v_swappgsout: 0
vm.stats.vm.v_swappgsin: 0
vm.stats.vm.v_swapout: 0
vm.stats.vm.v_swapin: 0
vm.swap_idle_threshold2: 10
vm.swap_idle_threshold1: 2
vm.nswapdev: 1
vm.swap_async_max: 4
vm.swap_maxpages: 64926752
vm.swap_reserved: 10847272960
vm.swap_total: 5368709120

I think swap is enabled, but I'm not sure if it's enabled properly... I never see anything in it...

Also there was the ZFS ARC that was set to take up to 14 GB of my 16... and it didn't feel nimble... I just let the system run all night after a reboot, and there it was this morning, using all it could... : \

I'll see if that helps... hopefully I don't have any bad RAM...
 

sidetone

Daemon

Reaction score: 832
Messages: 1,728

I think swap is enabled, but I'm not sure if it's enabled properly... I never see anything in it...

Also there was the ZFS ARC that was set to take up to 14 GB of my 16... and it didn't feel nimble... I just let the system run all night after a reboot, and there it was this morning, using all it could... : \

I'll see if that helps... hopefully I don't have any bad RAM...

swapinfo -m

Swap is enabled through fstab(5). There are a few examples in that manpage, some with encryption. If you don't have a dedicated hard drive partition for swap, you can use a filename for it, but that will wear out parts of your hard drive faster.

A book on FreeBSD would help you a lot, but you can also learn a lot without it.
 

rigoletto@

Daemon
Developer

Reaction score: 1,249
Messages: 2,291

If you don't have a dedicated hard drive partition for swap, you can use a filename for it, but that will wear out parts of your hard drive faster.

This is specially bad when using ZFS, it will work but will bring more memory issues.

A book on FreeBSD would help you a lot, but you can also learn a lot without it.

Absolute FreeBSD.
 
OP
poorandunlucky

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

swapinfo -m

Swap is enabled through fstab(5). There are a few examples in that manpage, some with encryption. If you don't have a dedicated hard drive partition for swap, you can use a filename for it, but that will wear out parts of your hard drive faster.

A book on FreeBSD would help you a lot, but you can also learn a lot without it.

Yeah, swap is enabled... idk why it's never used, though... probably because it waits to be out of RAM, as I'd expect... I have 16 GB of RAM, which is plenty for a single user, I only set 5 GB aside in case of overflow... Though I set it to swap out idling processes, though... though maybe there's just none...

Code:
Device          1M-blocks     Used    Avail Capacity
/dev/ada0p2.eli      5120        0     5120     0%

Code:
################################################################################
# DogeOS 0.0.1
# Filesystems Table
# ------------------------------------------------------------------------------
# Configures filesystems to mount at boot time, and to automount when available.
# ==============================================================================

# ====================================================================================================
# Device                Mountpoint              FStype          Options                 Dump    Pass
# ----------------------------------------------------------------------------------------------------
# NTFS Partition from old Windows installation (mounts read-only only).
# To be made into a more compatible filesystem as soon as possible.
#
#/dev/ada1s1            /mnt                    fuse.ntfs       ro,noauto               0       0
#
# Swap space.
#
/dev/ada0p2.eli         none                    swap            sw                      0       0
#
# procfs
#
#proc                   /var/proc               procfs          rw                      0       0
#
# ====================================================================================================

# ==============================================================================
# DogeOS 0.0.1 - Yoonix | poorandunlucky <poorandunlucky@boardermail.com>
################################################################################

Yes, I have very neat configuration files, I know :p
 

Beastie

Daemon

Reaction score: 542
Messages: 2,275

Yeah, swap is enabled... idk why it's never used, though... probably because it waits to be out of RAM, as I'd expect... I have 16 GB of RAM, which is plenty for a single user, I only set 5 GB aside in case of overflow...
Yes, swap is not used when it's not needed. My machine has 8GB and it rarely swaps even when compiling or copying hundreds of GBs of data.


I still don't know what preemption scheduling is or means...
Preemptive scheduling is a type of process management algorithm. It simply means that the kernel allocates execution time to each process and interrupts the execution of the currently-running process when its time's up. How that is done depends on the kernel type and the specific algorithm chosen for its process manager.
This sysctl tunable is just a parameter that tweaks the behavior of the scheduler.
 

Snurg

Daemon

Reaction score: 589
Messages: 1,349

Back to topic...
Code:
sysctl kern.sched.preempt_thresh=224
My question is: What does it mean, and what does it do?
It obviously determines the threshold preemptive multitasking sets in, in case a process doesn't yield back cpu in time.
This reminds me of the old kernel option HZ=100, which many tuned to HZ=1000 or higher to reduce the latency.
I used it myself in FreeBSD 5, but nowadays it is no longer in GENERIC and only mentioned indirectly in NOTES. So I guess it is obsolete.
 
OP
poorandunlucky

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

Beastie, Snurg:

So this value determines how much time a job can run on the CPU before it's suspended, and other tasks in the queue can run (all depending on their niceness, and affinity, of course), and if they don't complete in the time that's given to them, they're also suspended, and back to the first suspended job, until all jobs complete... right?

Code:
kern.sched.topology_spec: <groups>
kern.sched.steal_thresh: 2
kern.sched.steal_idle: 1
kern.sched.balance_interval: 127
kern.sched.balance: 1
kern.sched.affinity: 1
kern.sched.idlespinthresh: 157
kern.sched.idlespins: 10000
kern.sched.static_boost: 152
kern.sched.preempt_thresh: 224
kern.sched.interact: 30
kern.sched.slice: 12
kern.sched.quantum: 94488
kern.sched.name: ULE
kern.sched.preemption: 1
kern.sched.cpusetsize: 32

These are the other tunables, for reference, if you guys wonder what else there is...

SirDice, (and others): Do you know if there's a document somewhere that lists and explains all the system variables and tunables (better than sysctl -ad)... something I could understand?
 

rigoletto@

Daemon
Developer

Reaction score: 1,249
Messages: 2,291

Trying to tweak too much always and up in slowness and mess.

This is my entire /etc/sysctl.conf
Code:
net.inet.ip.random_id=1
kern.coredump=0
hw.snd.default_unit=5
hw.snd.feeder_rate_quality=4
hw.snd.vpc_0db=100
hw.syscons.bell=0
kern.sched.preempt_thresh=224
kern.maxfiles=450000
kern.maxvnodes=250000
 
OP
poorandunlucky

poorandunlucky

Well-Known Member

Reaction score: 33
Messages: 372

Trying to tweak too much always and up in slowness and mess.

This is my entire /etc/sysctl.conf
Code:
net.inet.ip.random_id=1
kern.coredump=0
hw.snd.default_unit=5
hw.snd.feeder_rate_quality=4
hw.snd.vpc_0db=100
hw.syscons.bell=0
kern.sched.preempt_thresh=224
kern.maxfiles=450000
kern.maxvnodes=250000

What are vnodes?
 

PacketMan

Aspiring Daemon

Reaction score: 166
Messages: 958

My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive,.......

I actually have another problem semi-related to compiling in RAM with Poudriere... it runs out of memory, and it creates a kernel panic, and everything just stops working...

When I go to 'rebuild' any of my machines, I tell my family I am doing maintence. I edit /etc/rc.conf and comment out all the stuff that starts up including desktop. Reboot. Now I got a machine with not much running, just the OS and terminal sessions. I run ports-mgmt/synth, the compiling eats ram like crazy, still get some swapping, but the machine plows through it like a heavy-duty purring diesel engine. After I am done with the rebuilt I uncomment the stuff in edit /etc/rc.conf and reboot. Always ask yourself and think through: "What's the problem you are trying to fix?".

Just a thought. :)
 

azathoth

Well-Known Member

Reaction score: 14
Messages: 380

When looking at how to tweak FreeBSD for desktop use, the most common tweak is this:

Code:
sysctl kern.sched.preempt_thresh=224
Default: 80

This, supposedly, does something with X Windows, allowing it to run more smoothly when CPU use is very high... or something like that...

My question is: What does it mean, and what does it do?

Also, I can't set it in /boot/loader.conf... how do I set it?

Also, any things you'd recommend doing, before a custom kernel, to optimize FreeBSD for desktop use? Anything you did you want to share?

I know of tuning(7), but it's not exactly a stimulating read... : <


Who is the guy in your profile pic?
 

wolffnx

Aspiring Daemon

Reaction score: 230
Messages: 675

Beastie, Snurg:

So this value determines how much time a job can run on the CPU before it's suspended, and other tasks in the queue can run (all depending on their niceness, and affinity, of course), and if they don't complete in the time that's given to them, they're also suspended, and back to the first suspended job, until all jobs complete... right?

Code:
kern.sched.topology_spec: <groups>
kern.sched.steal_thresh: 2
kern.sched.steal_idle: 1
kern.sched.balance_interval: 127
kern.sched.balance: 1
kern.sched.affinity: 1
kern.sched.idlespinthresh: 157
kern.sched.idlespins: 10000
kern.sched.static_boost: 152
kern.sched.preempt_thresh: 224
kern.sched.interact: 30
kern.sched.slice: 12
kern.sched.quantum: 94488
kern.sched.name: ULE
kern.sched.preemption: 1
kern.sched.cpusetsize: 32

These are the other tunables, for reference, if you guys wonder what else there is...

SirDice, (and others): Do you know if there's a document somewhere that lists and explains all the system variables and tunables (better than sysctl -ad)... something I could understand?

I' changed

Code:
kern.sched.slice to 3
and kern.sched.interact to 5

and a notice a little speedup when starting applications
 
Top