HOWTO: FreeBSD with CCACHE

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

install ccache port or add a package:
Code:
# cd /usr/ports/devel/ccache && make install clean
# pkg_add -r ccache
add the following to [font="Courier New"]/etc/make.conf[/font]:
Code:
.if !defined(NO_CCACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CXX= /usr/local/libexec/ccache/world-c++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif
add the following to [font="Courier New"]/.cshrc[/font]:
Code:
# set ccache varibles
setenv PATH /usr/local/libexec/ccache:$PATH
setenv CCACHE_PATH /usr/bin:/usr/local/bin
setenv CCACHE_DIR /var/tmp/ccache
setenv CCACHE_LOGFILE /var/log/ccache.log

# set ccache temp size to 512MB (default 1GB)
if ( -x /usr/local/bin/ccache ) then
  /usr/local/bin/ccache -M 512m > /dev/null
endif
logout and login again before using ccache because when You dont ccache will use [font="Courier New"]/root/.ccache[/font] dir instead of [font="Courier New"]/var/tmp/ccache[/font].

ccache can be shared between several computers the same as ports tree for example, check man ccache for more info.

ADDED 2007.2.15:
example ccache stats from my box.
Code:
% ccache -s
cache directory                     /var/tmp/ccache
cache hit                          18562
cache miss                        102820
called for link                     9824
multiple source files                 75
compile failed                      1610
preprocessor error                  1446
not a C/C++ file                    3747
autoconf compile/link              16982
unsupported compiler option          511
no input file                       6698
files in cache                     49631
cache size                         464.3 Mbytes
max cache size                     512.0 Mbytes
ADDED 2007.2.16:
comparasion of buildworld times with and without ccache:
Code:
# without ccache
make -j1 buildworld  4148.38s user 937.02s system 97% cpu [B]1:27:00[/B].40 total

# with ccache
make -j1 buildworld  1043.30s user 703.76s system 88% cpu [B]32:50[/B].02 total
If you face a problem with [font="Courier New"]/root/.ccache[/font] being used instead of [font="Courier New"]/var/tmp/ccache[/font] then you may solve it that way:

Code:
% cd /home/${USER}
% rm -rf .ccache
% ln -s /var/tmp/ccache .ccache
% ls -l .ccache
lrwxr-xr-x  1 ${USER}  ${USER}  15 Dec 19 15:20 .ccache -> /var/tmp/ccache
% ccache -s
cache directory                     /var/tmp/ccache
cache hit                         165292
cache miss                        327142
called for link                    38002
multiple source files                216
compile failed                      5182
preprocessor error                  4934
couldn't find the compiler             1
not a C/C++ file                   26249
autoconf compile/link              52665
unsupported compiler option         1379
no input file                      23289
files in cache                     79438
cache size                         530.2 Mbytes
max cache size                     512.0 Mbytes
% sudo ccache -s
cache directory                     /home/vermaden/.ccache
cache hit                         165292
cache miss                        327142
called for link                    38002
multiple source files                216
compile failed                      5182
preprocessor error                  4934
couldn't find the compiler             1
not a C/C++ file                   26249
autoconf compile/link              52665
unsupported compiler option         1379
no input file                      23289
files in cache                     79438
cache size                         530.2 Mbytes
max cache size                     512.0 Mbytes
 

kamikaze

Well-Known Member

Reaction score: 75
Messages: 365

<shameless advertising>
I'm using buildflags from sysutils/bsdadminscripts to do that. It also deals with using both, ccache and distcc at the same time.
</shameless advertising>

I have come to the conclusion that the default cache size of 1GB is way too low. At the moment I'm using 4GB:
Code:
# ccache -s
cache directory                     /root/.ccache
cache hit                         607814
cache miss                        993734
called for link                    76913
multiple source files                316
compile failed                     11642
preprocessor error                  7718
couldn't find the compiler             4
not a C/C++ file                   90356
autoconf compile/link              97184
unsupported compiler option        14339
no input file                      29725
files in cache                    269281
cache size                           3.6 Gbytes
max cache size                       4.0 Gbytes
As you can see I have a hit rate of ~38%, which is a pretty impressive number, I think. I don't know how mature that previous example is (ccache has to be in use for some time to bring benefits), but the shown 512MB cache only has a hit rate of ~15%
 
OP
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

kamikaze said:
<shameless advertising>
I'm using buildflags from sysutils/bsdadminscripts to do that. It also deals with using both, ccache and distcc at the same time.
</shameless advertising>

I have come to the conclusion that the default cache size of 1GB is way too low. At the moment I'm using 4GB (...) As you can see I have a hit rate of ~38%, which is a pretty impressive number, I think. I don't know how mature that previous example is (ccache has to be in use for some time to bring benefits), but the shown 512MB cache only has a hit rate of ~15%
Thanks for tips, propably it was used for short period of time, the last stats from my post give these stats with 512MB size:
Code:
cache hit                         165292
cache miss                        327142
About 33% so not so bad.
 

kamikaze

Well-Known Member

Reaction score: 75
Messages: 365

More than 30% is very decent. Compiling OpenOffice would blow your cache apart, though. Everything useful would be flushed out. OpenOffice is the reason I stepped from 1GB to 4GB.
 
OP
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

Compiling OpenOffice is like suicide, I always use packages for such big blobs like that ;)
 

kamikaze

Well-Known Member

Reaction score: 75
Messages: 365

You normally have to wait for a long time for new OOo packages. That's why I build them myself and share them.
 
OP
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

Propably I do not use OpenOffice that much to care about newest version, current 2.4 version seems to work fine.

Where do you keep your builds mate?

Maybe I will try them some day.
 

Weaseal

Member

Reaction score: 8
Messages: 56

Is there an ad-hoc way to disable CCACHE? For example, something like:
Code:
make install clean WITHOUT_CCACHE=yes
 
OP
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

like that mate:
[cmd=]# make NO_CACHE=yes install clean[/cmd]
 

kegf

New Member

Reaction score: 1
Messages: 12

change
vermaden said:
Code:
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CCX= /usr/local/libexec/ccache/world-cc++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif
to
Code:
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CCX= /usr/local/libexec/ccache/world-c++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif
 

Mel_Flynn

Well-Known Member

Reaction score: 81
Messages: 379

Add a note:
You have to reinstall devel/libtool15 if it's installed, as changing CC/CXX will screw things up for it.

Adding the PATH change is not required and can create problems with lang/gcc* ports as dependencies, as it will add gcc## to $PATH and BUILD_DEPENDS will think it's installed, but once compilation starts it doesn't work.

I use:
:setenv=....<stripped other stuff>,CCACHE_DIR=/var/db/ccache/$:

in /etc/login.conf to make CCACHE_DIR independant of shell semantics and dot.profile files.

Finally the stats from my package builder:
Code:
cache directory                     /var/db/ccache/root
cache hit                         508615
cache miss                        379447
called for link                    66381
multiple source files                228
compile failed                     13214
ccache internal error                  2
preprocessor error                 21943
not a C/C++ file                   27067
autoconf compile/link             118453
unsupported compiler option         6337
no input file                      38147
files in cache                    650587
cache size                           8.6 Gbytes
max cache size                      10.0 Gbytes
 

dennylin93

Aspiring Daemon

Reaction score: 109
Messages: 783

vermaden said:
Code:
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CCX= /usr/local/libexec/ccache/world-c++
.endif
A small question. It is CCX or CXX? In /usr/local/share/doc/ccache/ccache-howto-freebsd.txt, is's CXX.
 

Artefact2

Member


Messages: 23

Thanks for this tip.

A little precision : if you use a non-csh shell, you have to edit the rc file of your shell, not /.cshrc. The syntax might change (export instead of setenv, ...).
 
OP
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

@Artefact2

That's obvious, but thanks for pointing it out.
 

graudeejs

Son of Beastie

Reaction score: 694
Messages: 4,615

My Stats:

I build OpenOffice.org-3.1 uncached (with ccache on) in about 10-11 hours

It took about 3h 40min to compile OOO3.1 when it was cached in ccache :)

My PC:
Pentium 4 HTT @ 3GHz
RAM: 2.5GB @ 400Mhz
on GELI encrypted zpool :)

Code:
killasmurf86 $ ccache -s
cache directory                     /var/db/ccache
cache hit                          79143
cache miss                        139100
called for link                    14059
multiple source files                 75
compile failed                      3599
ccache internal error                  1
preprocessor error                  2494
couldn't find the compiler             2
not a C/C++ file                    7095
autoconf compile/link              28680
unsupported compiler option          932
no input file                      10850
files in cache                    278200
cache size                           2.6 Gbytes
max cache size                       4.0 Gbytes
 

nal

New Member

Reaction score: 3
Messages: 13

Code:
# Allow CCACHE for PORTS only (the buildworld has error with CCACHE)
.if ${.CURDIR:M*/ports*}
.if exists(/usr/local/libexec/ccache/world-cc) && !defined(NO_CCACHE)
CC=/usr/local/libexec/ccache/world-cc
CXX=/usr/local/libexec/ccache/world-c++
.endif
CFLAGS= ...
CXXFLAGS=${CFLAGS}
...
...
.endif
 

Seeker

Aspiring Daemon

Reaction score: 17
Messages: 865

vermaden said:
like that mate:
[cmd=]# make NO_CACHE=yes install clean[/cmd]
A lots of luniz here will just copy-paste it and hit a wall:
Code:
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CXX= /usr/local/libexec/ccache/world-c++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_[color="Red"][B]C[/B][/color]CACHE= yes
.endif
 
OP
OP
vermaden

vermaden

Son of Beastie

Reaction score: 1,181
Messages: 2,764

@Seeker

Thanks, its also funny how long it was here without notifying ;)
 

mfaridi

Aspiring Daemon

Reaction score: 15
Messages: 713

my apace of /var is limit and I use this command
Code:
portmaster -D -r -w gettext
and I see ccache take many space of
Code:
/var
what happen when I delete all files in ccache folder , because I do not have many space ?
 

dennylin93

Aspiring Daemon

Reaction score: 109
Messages: 783

Simply limit the size of your cache using # ccache -M

If you delete the files, your cache will be empty.
 

Slade

New Member

Reaction score: 9
Messages: 19

I'm wondering how I would go about using a new version of gcc (gcc45) with ccache. Right now I have just gcc45 running per the instructions at: http://www.freebsd.org/doc/en/articles/custom-gcc/article.html

That works fine but how would I go about getting ccache to use that. It seems like the instructions are for the standard version of gcc that comes with FreeBSD.

My current /etc/make.conf
Code:
.if !empty(.CURDIR:M/usr/ports/*) && exists(/usr/local/bin/gcc45)
CC=gcc45
CXX=g++45
CPP=cpp45
CPUTYPE?=amd64
.endif
my current /etc/libmap.conf
Code:
libgcc_s.so.1   gcc45/libgcc_s.so.1
libgomp.so.1    gcc45/libgomp.so.1
libobjc.so.3    gcc45/libobjc.so.2
libssp.so.0     gcc45/libssp.so.0
libstdc++.so.6  gcc45/libstdc++.so.6
I've tried looking over the examples in this thread and the ccache-howto-freebsd.txt when I installed ccache from ports but I just don't know enough about the make process to figure out how to get it working with gcc45.
 
Top