Kernel tuning in FreeBSD

[fateh]

Hi Team

I am doing kernel tuning in FreeBSD, I need your support on leveraging kernel tuning parameters from NetBSD. Below are the parameters from NetBSD we already have in our environment.

< maxusers 256 # estimated number of users
---
> maxusers 64 # estimated number of users
56c56
< #options INSECURE # disable kernel security levels - X needs this
---
> options INSECURE # disable kernel security levels - X needs this
108,113d107
< options KMEMSTATS
< options MAXUPRC=4800
< options SHMMAXPGS=2097152
< options SEMMNI=256
< options SEMMNS=2048
< options SEMMAP=4096
227c221
< options GATEWAY # packet forwarding
---
> #options GATEWAY # packet forwarding
268d261
< pseudo-device ipfilter # IP filter (firewall) and NAT

I want to leverage similar parameters in FreeBSD. So Far I created cp /usr/src/sys/amd64/conf/GENERIC /usr/src/sys/amd64/conf/VAULTS.
I am trying to find similar type of parameters in FreeBSD. If anyone can help and guide me for this would be highly appreciated.

If I run with the above options in FreeBSD, it says unknown option "INSECURE" May be FreeBSD different options similar to this.

Help will be highly appreciated.

Thanks,
Fateh

 

[SirDice]

I need your support on leveraging kernel tuning parameters from NetBSD.

I hope you realize the FreeBSD and NetBSD kernels have diverged quite a lot over the years? Why are you trying to apply the same "tuning"?

maxusers can be dynamically changed, no need for compile-time kernel settings. It's set at a specific number based on the amount of memory the machine has.

root@chibacity:~ # sysctl -d kern.maxusers
kern.maxusers: Hint for kernel tuning

INSECURE is not an option you can set, just don't set a securelevel(7).

kern_securelevel="-1"   # range: -1..3 ; `-1' is the most insecure
                        # Note that setting securelevel to 0 will result
                        # in the system booting with securelevel set to 1, as

The various shared memory options are all dynamically changeable. They're not compile-time kernel options.

kern.ipc.shmall: 131072
kern.ipc.shmseg: 128
kern.ipc.shmmni: 192
kern.ipc.shmmin: 1
kern.ipc.shmmax: 536870912

GATEWAY doesn't need to be compiled in. Just add gateway_enable="YES" to rc.conf to turn it on. For IPv6 it's ipv6_gateway_enable="YES"

PF can be loaded as a module, pf(4), no need to bake it in. It will be loaded automatically when you enable the firewall in rc.conf; pf_enable="YES".

I want to leverage similar parameters in FreeBSD.

Don't "tune" for the sake of tuning. Tune because you need to tweak some settings. FreeBSD does an excellent job tuning itself for most situations. I suggest you simply start with the GENERIC kernel and see what that gets you.

 

[covacat]

also man 7 tuning
a lot can be done without recompiling/rolling your own

 

[cy@]

This is so 90's.

 

[fateh]

I hope you realize the FreeBSD and NetBSD kernels have diverged quite a lot over the years? Why are you trying to apply the same "tuning"?

maxusers can be dynamically changed, no need for compile-time kernel settings. It's set at a specific number based on the amount of memory the machine has.

root@chibacity:~ # sysctl -d kern.maxusers
kern.maxusers: Hint for kernel tuning

INSECURE is not an option you can set, just don't set a securelevel(7).

kern_securelevel="-1"   # range: -1..3 ; `-1' is the most insecure
                        # Note that setting securelevel to 0 will result
                        # in the system booting with securelevel set to 1, as

The various shared memory options are all dynamically changeable. They're not compile-time kernel options.

kern.ipc.shmall: 131072
kern.ipc.shmseg: 128
kern.ipc.shmmni: 192
kern.ipc.shmmin: 1
kern.ipc.shmmax: 536870912

GATEWAY doesn't need to be compiled in. Just add gateway_enable="YES" to rc.conf to turn it on. For IPv6 it's ipv6_gateway_enable="YES"

PF can be loaded as a module, pf(4), no need to bake it in. It will be loaded automatically when you enable the firewall in rc.conf; pf_enable="YES".


Don't "tune" for the sake of tuning. Tune because you need to tweak some settings. FreeBSD does an excellent job tuning itself for most situations. I suggest you simply start with the GENERIC kernel and see what that gets you.

Thanks for your overall evaluation on, now it make sense what we can do and what we can't.

We can apply these changes dynamically through command line or we can specify this sysctl.conf and reboot?

 

[fateh]

The question that arise in my mind is lets say if we have 200 to 300 freebsd servers, how do we manage tunings? how does the big organizations handlining tuning in FreeBSD? if we put all of these in the kernel then we don't have to go and do this steps for 200 to 300 servers.
Is there any way we can build this parameters from kernel by compiling?

 

[T-Aoki]

Thanks for your overall evaluation on, now it make sense what we can do and what we can't.

We can apply these changes dynamically through command line or we can specify this sysctl.conf and reboot?

It depends.
There are 2 attributes with single mechanins, sysctls and tunables.
The developer implementing new one can choose

• tunable only,
• sysctl only or
• allow both.

Tunable can be set on /boot/loader.conf[.local] (means, set before kernel starts),
sysctl can be set on command line and/or /etc/sysctl.conf (means, set after kernel starts).

Tunables which does not allowed to be sysctl are sometimes readonly after boot, thus, need reboot to change the value. This typically appears as something defining maximum allowed resources which can only configured before kernel starts.

 

[ralphbsz]

The question that arise in my mind is lets say if we have 200 to 300 freebsd servers, how do we manage tunings? how does the big organizations handlining tuning in FreeBSD?
Is there any way we can build this parameters from kernel by compiling?

How do big organizations manage tuning (anything, whether it be the OS kernel, libraries, applications such as databases, network stacks, ...)? Measure, measure, and measure more, while carefully varying tuning parameters. Then analyze the measurements, and build models that describe how objectives (such as performance or cost or reliability or ...) will depend on parameters. Talk to experts to make sure the models make sense, and we are not optimizing for spurious correlations. Then optimize. Set the optimal parameters, and measure more. Keep tracking the objectives, and keep updating the models.

Most of these parameters make very little difference. Typically single digit percent. In an organization that has a million servers, it makes sense to invest time and effort (meaning lots of $$$) to optimize such small effects, as saving 10,000 servers pays for someone's salary. An old colleague of mine used to boast that within a year of being hired by <name of big internet company> in the early 2010s, he saved them a billion dollars, which got him a promotion. For a person who deployed 1 or 2 servers, it usually makes no sense to invest the effort.

NEVER EVER TUNE ANYTHING WITHOUT MEASURING AND UNDERSTANDING FIRST. The defaults are there for a reason.

if we put all of these in the kernel then we don't have to go and do this steps for 200 to 300 servers.

If you deploy hundreds (or thousands or millions) of machines, the one file /etc/sysctl.conf is the least of your problems. Dozens or hundreds of other things need to be adjusted to deploy a machine. Doing a custom kernel just to avoid one file is a really bad tradeoff. There may be other reasons for a large organization to use custom kernels, but this one alone isn't it.

 

[SirDice]

The question that arise in my mind is lets say if we have 200 to 300 freebsd servers, how do we manage tunings?

The better question is, how to manage 200-300 servers at all. Do you think those are managed by hand? Someone logging in, installing and configuring software, then checking everyday? No, of course not. Ansible, Puppet, or even custom scripts. If you manage the software on those machines you can also manage any tuning that might be required. And you have a whole bunch of monitoring software to measure statistics, keep an eye on broken hardware, etc.

Is there any way we can build this parameters from kernel by compiling?

Think about this for a second, this also means you're going to need to distribute custom kernels everywhere. You think that'll be easier than tuning some parameters?

 

[fateh]

The better question is, how to manage 200-300 servers at all. Do you think those are managed by hand? Someone logging in, installing and configuring software, then checking everyday? No, of course not. Ansible, Puppet, or even custom scripts. If you manage the software on those machines you can also manage any tuning that might be required. And you have a whole bunch of monitoring software to measure statistics, keep an eye on broken hardware, etc.


Think about this for a second, this also means you're going to need to distribute custom kernels everywhere. You think that'll be easier than tuning some parameters?

Not sure why he wants to go against system architecture and making things more complicated. Even though I explained him that if you are thinking to manage 200 to 300 servers, we have Ansible and many other tools in place to manage these all tunings. Can't explain this guy anymore. Literally it sucks me now. Never seen such an unprofessional manager who force to do this else want to replace me. kind of threatening me.