HOWTO: FreeBSD with CCACHE

camelia

Member

Reaction score: 8
Messages: 43

BTW, I'd suggest using `?=' everywhere. It leaves the chance to override value from environ(7) without touching make.conf.

As for port's gcc there are few pitfalls but libmap.conf addresses only runtime linking problem completely avoiding build linking one (rtld vs. ld). BTW, using CPUTYPE?=native should work for any GCC (newer than gcc3*) and clang. If you're after more portable binaries then you'd want to use -mtune=generic but there is no make(1) macro for it.
 

Seeker

Aspiring Daemon

Reaction score: 17
Messages: 865

I have 2 questions:

1) Why exactly, do we have to recompile, devel/libtool22, after we change CC and CXX? What are the consequences, of not doing so?

2) WITH ccache, I did a buildkernel and it took 43 min, to finish. Without ccache it took 39 min, to finish.
Then mistakenly, I've issued a same command again and ONLY then it did compiled in 9 minutes!
So, this means that ccache, speeds up RE-compilation, not a initial compilation, which can be even slower!
Now, as far as I know for myself, after build, I go for install.
I can't imagine, even one scenario, where I would go and build already build thing once again.
 

wblock@

Beastie Himself
Developer

Reaction score: 3,718
Messages: 13,851

Seeker said:
2) WITH ccache, I did a buildkernel and it took 43 min, to finish. Without ccache it took 39 min, to finish.
Then mistakenly, I've issued a same command again and ONLY then it did compiled in 9 minutes!
So, this means that ccache, speeds up RE-compilation, not a initial compilation, which can be even slower!

Same as a browser cache. The first time you go to that page, it's a little slower because it has to cache everything. Second time, it only retrieves the things that have changed, so it's faster. Yes, there's some added overhead in saving the cached files, but it's usually faster than retrieving them again.

Now, as far as I know for myself, after build, I go for install.
I can't imagine, even one scenario, where I would go and build already build thing once again.

When you buildworld or kernel, odds are good that a lot of it hasn't changed since the last time it was built, even weeks or months ago. How much has changed depends on how long it's been since you last built world. Next time you buildworld, time it again. If it takes less than 35 minutes (39 minutes normal minus the four already lost to overhead)... you win! After that, anything less than 39 minutes is pure win.
 

Seeker

Aspiring Daemon

Reaction score: 17
Messages: 865

wblock said:
When you buildworld or kernel, odds are good that a lot of it hasn't changed since the last time it was built, even weeks or months ago. How much has changed depends on how long it's been since you last built world. Next time you buildworld, time it again. If it takes less than 35 minutes (39 minutes normal minus the four already lost to overhead)... you win! After that, anything less than 39 minutes is pure win.
Well I did timed it here and after initial kernel compilation of 43 min, RE-compilation was done in 9 min.
Which is almost, 5 times better, performance boost.

Even few months?!
In that time frame, I would do many, many compilations, of various ports, etc...
Perhaps, a dedicated HDD, just for CCACHE_DIR /var/tmp/ccache, would suffice? :p
 
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,572
Messages: 2,996

@Seeker

These are mine ccache stats after about 12 months of usage:
Code:
% zfs list basefs/var/ccache
NAME                USED  AVAIL  REFER  MOUNTPOINT
basefs/var/ccache   575M   569G   575M  /var/ccache

% ccache -s
cache directory                     /var/ccache
cache hit                          17306
cache miss                         43982
called for link                     5148
multiple source files                  5
compile failed                      3107
preprocessor error                   819
not a C/C++ file                    3423
autoconf compile/link               3957
unsupported compiler option         1221
no input file                       1343
files in cache                     87964
cache size                           1.1 Gbytes
max cache size                       4.0 Gbytes

Also, if You limit it to 1GB in config, then it will STAY 1GB size.
 

ohauer

Active Member

Reaction score: 17
Messages: 127

Seeker said:
I can't imagine, even one scenario, where I would go and build already build thing once again.

Hm, think about you maintain some ports and you will test all different build options
or you patch parts of the source, generate the patchfiles and build again ...

With usage of CCACHE the configure scripts are the main bottleneck for me, and I have similar values like vermaden.
 

camelia

Member

Reaction score: 8
Messages: 43

ohauer said:
think about you maintain some ports and you will test all different build options
or you patch parts of the source, generate the patchfiles and build again
Most of the time changing configure options affects config.h or CFLAGS that cause cache misses during build.
 

Beeblebrox

Aspiring Daemon

Reaction score: 148
Messages: 985

ccache with GCC45

I am using gcc45 for ports, native (gcc42) for source. Therefore the conditional statement in make.conf needs to be a little different for my setup. After some debugging, I seem to have found the correct settings for my make.conf
Code:
.if !empty{.CURDIR:M/usr/ports/*}
CC=  /usr/local/libexec/ccache/world[color="Red"][B]/gcc45[/B][/color]
CXX= /usr/local/libexec/ccache/world[color="red"][B]/g++45[/B][/color]
CFLAGS+= -mssse3
.else 
CC=  /usr/local/libexec/ccache/world[color="Red"][B]/cc[/B][/color]
CXX= /usr/local/libexec/ccache/world[color="red"][B]/c++[/B][/color]
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
  CC= gcc45
  CXX= g++45
  CPP= cpp45
  CFLAGS+= -mssse3
.endif
Check your /usr/local/libexec/ccache/world/ directory and make sure you have all of the above named files in the folder. These of course, are not files but soft-links to the ccache file in the same folder. If you do not have such links, create them (#ln -s) with the appropriate names specified here (ccache should have created them already though).

Whenever you change (upgrade) ccache versions or add a different compiler (say you switched form 4.5 to 4.6 or such) you must re-build devel/libtool, otherwise you will see "precompiler fails sanity check" messages.
 

qsecofr

Active Member

Reaction score: 15
Messages: 246

I needed to edit /etc/make.conf as suggested by Beeblebrox:
Code:
CC=  /usr/local/libexec/ccache/world[B][color="Red"]/[/color][/B]cc
  CXX= /usr/local/libexec/ccache/world[B][color="Red"]/[/color][/B]c++
The original suggestion had dashes instead of forward slashes. Not sure if the original was a typo or worked as intended on author's system. I'm using FreeBSD 8.1-Release.

Also replies suggested building libtool15. I have a more recent version. Not sure if that needs rebuilding, but I did. Was there anything version-specific about that recommendation?

It would have been good if the HOWTO referenced /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and explained any differences.

I agree with Seeker in that a distinction exists between compilation vs. re-compilation. I have no idea how many times I re-compile the same version of the same port. Not by intent anyway - I leave that decision up to portmaster and or make. But I'm hopeful the tool has finer granularity than just overall port version. I'll probably know the next time I upgrade KDE. Maybe.
 

wblock@

Beastie Himself
Developer

Reaction score: 3,718
Messages: 13,851

qsecofr said:
I agree with Seeker in that a distinction exists between compilation vs. re-compilation. I have no idea how many times I re-compile the same version of the same port. Not by intent anyway - I leave that decision up to portmaster and or make. But I'm hopeful the tool has finer granularity than just overall port version. I'll probably know the next time I upgrade KDE. Maybe.

Benchmark it. Set NO_CCACHE and time a build. Reboot to clear the filesystem cache, leave ccache enabled, and build it again. My guess for port updates is that ccache usually won't have it in the cache and will actually make port builds a little slower.

/usr/src doesn't change all that much from week to week, but there can still be a lot of cache misses caused by small changes.
 
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,572
Messages: 2,996

wblock said:
My guess for port updates is that ccache usually won't have it in the cache and will actually make port builds a little slower.
CCACHE is useful if you compile something at least TWO TIMES; the first time it's slower because it 'learns', the second and every other compile run is a lot faster. Not the first one.
 

wblock@

Beastie Himself
Developer

Reaction score: 3,718
Messages: 13,851

vermaden said:
CCACHE is useful if you compile something at least TWO TIMES; the first time it's slower because it 'learns', the second and every other compile run is a lot faster. Not the first one.

Of course. But do you think the cached version of KDE will still be in the cache by the time the next version comes out? And how much of it will be valid because the source hasn't changed? If it's not in the cache or invalid, the overhead of ccache will slow you down, and I suspect that will be the case most of the time with ports. Maybe a really big cache size would help. ccache -s will tell the story.
 
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,572
Messages: 2,996

I would say that if you compile a lot and have a quite large ccache space (4-8G) then the more you compile, the faster it will be, at least I got that impression after longer usage. Currently I only use packages and compile as little as possible, so I do not even bother with setting up CCACHE anymore.
 

bbzz

Aspiring Daemon

Reaction score: 134
Messages: 874

I have a habit of compiling everything as soon as it's updated. When I'm building as root it works. When I'm building as user using sudo it never uses CCACHE.

The configuration for root is in ~/.cshrc. The configuration for user is in ~/.zshrc. The configuration is identical, minus shell semantics, and both point to same CCACHE location, /var/db/ccache. So the both $PATH are equal.

Since I use sudo with user to update ports and build, CCACHE is never used. What do I need to do to make this work?

p.s. great guide as always.
 
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,572
Messages: 2,996

Edit the /usr/local/etc/sudoers file and add needed variables that should be kept when using sudo(8), for example:
Code:
Defaults        env_keep += "HOME PKG_PATH PKG_DBDIR PKG_TMPDIR TMPDIR PACKAGEROOT PACKAGESITE PACKAGES PKGDIR FTP_PASSIVE_MODE"
 

bbzz

Aspiring Daemon

Reaction score: 134
Messages: 874

Tried, but that doesn't help.
There is no update when I check cache with:
# ccache -s
Whereas, when run as root it get updated.

EDIT: I just realized that when using sudo user uses /root/.ccache.
EDIT2: Hence, fixed! Thx.
 

graudeejs

Son of Beastie

Reaction score: 704
Messages: 4,617

That's because If you use shared ccache you'll probably have permissions issues
 

bbzz

Aspiring Daemon

Reaction score: 134
Messages: 874

I changed permissions there. For some reason it only works when configuration is defined in both my user's ~/.zshrc AND settings forwarded in visudo.
If it's only configured in ~/.zshrc, but not in visudo (CCACHE_PATH, CCACHE_DIR), it uses default /root/.ccache location. If it's only defined in root's ~/.cshrc and forwarded with visudo, (so nothing defined in user's ~/.zshrc), then it uses user's default location, ~/home/$user/.ccache.
I don't get this, but I got it to work.
 

graudeejs

Son of Beastie

Reaction score: 704
Messages: 4,617

vermaden said:
install ccache port or add a package:
Code:
.if !defined(NO_CCACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CXX= /usr/local/libexec/ccache/world-c++
.endif

This should be updated to

Code:
.if !defined(NO_CCACHE)
  CC=  /usr/local/libexec/ccache/world/cc
  CXX= /usr/local/libexec/ccache/world/c++
.endif
 

wblock@

Beastie Himself
Developer

Reaction score: 3,718
Messages: 13,851

The port suggests
Code:
.if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
.if !defined(NOCCACHE)
CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
.endif
.endif
 

graudeejs

Son of Beastie

Reaction score: 704
Messages: 4,617

wblock@ said:
The port suggests
Code:
.if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
.if !defined(NOCCACHE)
CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
.endif
.endif

Yes, btw, I use
Code:
WRKDIRPREFIX=/usr/obj
When I used
Code:
if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj/usr/src*))
...
.endif
it (ccache) didn't work. Any idea why?
 

wblock@

Beastie Himself
Developer

Reaction score: 3,718
Messages: 13,851

That line uses ccache only for building the system. That's pretty reasonable, it's rare that ccache will help with building ports.
 

bbzz

Aspiring Daemon

Reaction score: 134
Messages: 874

Code:
.if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj/usr/src/*))
.if !defined(NOCCACHE)
CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
.endif
.endif

So this is the correct way to specify ccache only for world building and not ports? Is there an optimum size for ccache maybe?
 

wblock@

Beastie Himself
Developer

Reaction score: 3,718
Messages: 13,851

I've used sizes of 1G to 4G, and more really doesn't help. Either nothing major has changed in the source and it mostly comes from cache, or something major or minor has changed that invalidates all of the cache, and everything has to be rebuilt.
 
Top