question about speeding make buildworld up

B

bluephoenix

Hi all,

I have built world and kernel before and I know how time consuming it can get. So I want to know if the following two solutions can help speed it up safely by cutting on the amount of I/O taking place.

1. I thought of reducing the hard disk access by setting MAKEOBJDIRPREFIX to /tmp (provided it is a TMPFS). I have 4G of ram, so I think it is enough. Is this safe, or can it lead to it not being built correctly?

2. I know that make(1) rebuilds a target only if its dependencies are older than the target itself. So what about not making cleanworld after installing (provided I have enough space), and when I update the source (cvsup), the next make buildworld will rebuilt only the outdated binaries. However, the FreeBSD manual states that not making clean can lead to problems when updating (it does not get too specific).

So, should I put these methods into practice or not?

Thanks in advance.
 
Hi, from what I remember you can also speed up this process by removing the /usr/obj directory. Note that some files may have set schg flags, so remove it first.
ccache is good idea to (for more info, check; /usr/local/share/doc/ccache/ccache-howto-freebsd.txt). A good tutorial you can find here; HOWTO: FreeBSD with CCACHE (by vermanden).

For some info about optimizations, check; Optimizations - system-wide settings. It is related to the /etc/make.conf file. And manual of course; make.conf(5) (Remember one thing - every variable that can be placed in this file, will be used when you run the make command!)

Best regards!
 
The optimizations article does some things like custom CFLAGS which often cause problems and provide little benefit. When you have problems, expect the first question to be "did you have custom CFLAGS?"
 
My suggestion is don't use ccache.

A always fail with ccache when I run make buildworld. :(

I don't know why, but it just doesn't work for me.

However, ccache can speed up the kernel building process significantly. :e
 
True, wblock and DutchDaemon. I also think that CFLAGS setting is not needed. Simply use the defaults, because for the base system, really/simply they are not needed. But this does not mean that bluephoenix can not look for further information on this topic, right? On the other hand I only tried to show a few possibilities, which in some sense - I hope - can speed up this process.

And what about the -jX option, used sometimes to run with make command? Let see. If bluephoenix for example have multiple CPUs/cores in his computer, he can speed this process up by using e.g?;
# make -jX buildworld
Or maybe I am wrong and it lengthen the build process? Really, I did it (I mean build world/kernel etc) a long time ago, and some things I just forgot.

Summarizing: I bet on the removal of /usr/obj during the build process and (maybe) ccache use. That's my opinion.

[1] The X should contain the number of cores.
 
On my C2D dual-core system with ccache, I found that -j6 was the fastest. Building without ccache might change that. I always delete /usr/obj/usr before starting a buildworld.
 
What I did was: I made buildworld using ccache with /tmp/ccache (tmpfs) as the cache dir. I also used -j6 as wblock suggested (I have an Intel core 2 duo) and the build was succesful taking much less time to complete. I did not change cflags.

However, I still don't know whether I should make cleanworld (I have enough space to hold the object files) or keep them and therefore speed up the next build. What about the problems mentioned on the FreeBSD manual?

Any ideas?
 
bluephoenix said:
What I did was:
I made buildworld using ccache with /tmp/ccache (tmpfs) as the cache dir.
Click to expand...

ccache checks the cache before compiling a new version. If you don't keep that cache for next time, using ccache will just slow things down.

However ,I still dont know whether I should make cleanworld(I have enough space to hold the object files) or keep them and therefore speed up the next build.
Click to expand...

cleanworld is a pointless delay if you just delete /usr/obj before starting a buildworld. Your link is to the ports documentation, which has nothing at all to do with the operating system, and doesn't use /usr/obj anyway.
 
Benchmarking -j with devel/ccache:

Follow the port instructions to set up ccache. I also give it more disk space than default:
# ccache -M 4G

Build world once to fill the cache.

Testing:

# rm -rf /usr/obj/usr
# cd /usr/src && time make -j2 buildworld

Record the elapsed time.
Remove /usr/obj and build again with -j3, -j4, and so on until you get tired of it. Pick whichever value of -j resulted in the fastest compile.
 
Isn't ccache spoiling the -j benchmark?

As far as I get it, after the first test run, ccache's cache is already filled and ready to speed up subsequent builds. I think the second, third, fourth build will always be faster than the first one because ccache kicks in.

So, wouldn't it be better to benchmark without ccache?
 
You're responding to a thread from two years ago, but it's a valid question.

My impression is that higher numbers of jobs work better when ccache has most of the results cached. When the results mostly aren't in cache, it seems like fewer jobs may actually go quicker. You can test without removing ccache by setting NOCCACHE. That will keep the overhead the same.
 
You must log in or register to reply here.
Back
Top