Solved How to compile the kernel faster

Stefan Andritoiu

Stefan Andritoiu

I am writing some changes to the sched_ule.c file (only this file), and compiling the whole kernel like this
Code: 
# cd /usr/src
# make buildkernel KERNCONF=GENERIC
# make installkernel KERNCONF=GENERIC
Which takes about 10 minutes.
Is there a faster way of only compiling what is necessary and shortening the wait?
 
This is probably a bad advice for someone who's not familiar with compiling a kernel but you could comment out unnecessary drivers in the kernel config. I usually do that and it cuts down on compilation time. Only do it if you know which drivers your computer needs. If you are not sure then best place is to look in the boot-up log in /var/log/messages. Do take your time to figure out exactly which drivers your computer needs otherwise you may have problems booting it up.
 
The handbook says you can put MODULES_OVERRIDE = ... or WITHOUT_MODULES = ... in /etc/make.conf. Can you also build with the -j flag like with the buildworld target? I'm not sure, I've never looked into it.
 
wblock@ said:
Leave /usr/obj intact after a build. Then use make -DNO_CLEAN -j8 kernel KERNCONF=GENERIC. Only the new code will be compiled, all the rest has not changed and so make(1) skips it.

For -j, I use double the number returned by sysctl hw.ncpu.
Click to expand...

What happens if I mount /usr/obj and build related folders over a md device? do it really saves reads/writes and time?
 
ColdfireMC said:
What happens if I mount /usr/obj and build related folders over a md device? do it really saves reads/writes and time?
Click to expand...

If you mean using a md(4) ram disk for /usr/obj then don't. The /usr/obj directory is meant to be on permanent storage to be able to repeat the installation after a crash or a reboot. On top of that the make buildworld buildkernel procedure has never been disk I/O intensive, it is very much CPU intensive and a memory disk will not give much improvements in compilation times.
 
wblock@ said:
Leave /usr/obj intact after a build. Then use make -DNO_CLEAN -j8 kernel KERNCONF=GENERIC. Only the new code will be compiled, all the rest has not changed and so make(1) skips it.
Click to expand...

The switch for kernel builds is actually -DNO_KERNELCLEAN. However, even better is -DKERNFAST which expands to
-DNO_KERNELCONFIG -DNO_KERNELCLEAN -DNO_KERNELDEPEND -DNO_KERNELOBJ
 
ColdfireMC said:
What happens if I mount /usr/obj and build related folders over a md device? do it really saves reads/writes and time?
Click to expand...
The time savings comes almost all the object code already being there, so either the first build would take the full amount of time, or the contents would have to be copied from permanent storage. As kpa says, it would not save any time, and not really save any writes. I'm pretty sure I've benchmarked that, and it saved no time at all. To speed up compiles, either get a faster CPU, or reduce the amount of code that has to be compiled.
 
obsigna said:
The switch for kernel builds is actually -DNO_KERNELCLEAN. However, even better is -DKERNFAST which expands to
-DNO_KERNELCONFIG -DNO_KERNELCLEAN -DNO_KERNELDEPEND -DNO_KERNELOBJ
Click to expand...

Makefile.inc1 implies that NO_CLEAN applies during a kernel build, but I need to look at it more. I actually use a script to automate most of a buildworld/kernel, and adding some of those kernel options could speed up that part. A later part of that file shows that it can even be shortened by setting it to the config file name: KERNFAST=MYKERNEL.
 
Some numbers: on my first-generation Core i5-560m notebook with an SSD:

make -DNO_CLEAN -j8 kernel KERNCONF=I5 takes about 27 seconds.
make -j8 kernel KERNFAST=I5 takes about 15 seconds.
 
wblock said:
For -j, I use double the number returned by sysctl hw.ncpu.
Click to expand...
So if I type that command and get hw.ncpu: 8 as an output, it means I should build the kernel with the command make -j16 kernel?
 
oleglelchuk said:
. So if I type that command and get
hw.ncpu: 8 as an output, it means I should build the kernel with the command make -j16 kernel ?
Click to expand...
Only if your CPU supports hyperthreading AND hyperthreading is not disabled in the BIOS. Look at the output of:
dmesg | grep core.

At my box, 2 cores and hyperthreading enabled, the output is:
Code: 
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads
This means, I could run make -j4 kernel ... on my machine, 2 cores each running 2 independent threads, in total = 4.

I don't do this, because if I occupy my machine compiling the kernel using in parallel all available hyper-cores, it would be somewhat unresponsive on it's other duties.
 
I get this output from dmesg | grep core :
Code: 
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads 
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads

It means hyperthreading is enabled on my machine, right?
 
oleglelchuk said:
I get this output from dmesg | grep core :
Code: 
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads

It means hyperthreading is enabled on my machine, right?
Click to expand...

SMT is the shortcut for Simultaneous Multi-Threading, which is sort of the same as Intel's Hyper-Threading. In your case, SMT tells us, that most probably your system is not running on an Intel CPU, and in addition it got "only" 4 real cores that are enhanced by Multithreading to 8 hyper-cores. So, you want to use -j8 at max. However, ask yourself, if the difference to let's say -j4 is worth to put your machine partly offline. Try it!
 
obsigna said:
SMT is the shortcut for Simultaneous Multi-Threading, which is sort of the same as Intel's Hyper-Threading. In your case, SMT tells us, that most probably your system is not running on an Intel CPU, and in addition it got "only" 4 real cores that are enhanced by Multithreading to 8 hyper-cores. So, you want to use -j8 at max. However, ask yourself, if the difference to let's say -j4 is worth to put your machine partly offline. Try it!
Click to expand...

But my system is actually running on an Intel CPU. It's an i7-4790. And what's interesting is that after I upgraded the world and the kernel to the latest STABLE revision and rebooted, I get 3 lines, instead of 2, as an output from dmesg | grep core :
Code: 
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads

It's strange that what dmesg says now is different from what it said before.

Edit: Okay, that's weird. I rebooted the comp yet again, and now I get 4 lines as an output from this command, not 3! I don't understand why this is the case.
 
Just little addition.
As Remington and jrm wrote, comment out unnecessary drivers in the kernel config file and also compile only necessary kernel modules. In the /etc/make.conf
Code: 
MODULES_OVERRIDE =
It will decrease time for kernel compiling.

I always use classic method for compiling kernel. Example for i386 arch; kernel name is GENERIC.
1. First time
cd sys/i386/conf
config GENERIC
cd ../compile/GENERIC
make cleandepend
make depend
make
make install

2. After desired changes in thе kernel files just issue the make commands in kernel build directory
make
make install
It will compile only those files that have been changed.
 
Last edited by a moderator:
You must log in or register to reply here.
Back
Top