HOWTO: FreeBSD with CCACHE

Would you like to share some of your solutions for certain problems? Tips or tricks? Post here. All new topics are automatically moderated.

HOWTO: FreeBSD with CCACHE

Postby vermaden » 16 Nov 2008, 21:53

install ccache port or add a package:
Code: Select all
# cd /usr/ports/devel/ccache && make install clean
# pkg_add -r ccache


add the following to [font="Courier New"]/etc/make.conf[/font]:
Code: Select all
.if !defined(NO_CCACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CXX= /usr/local/libexec/ccache/world-c++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif


add the following to [font="Courier New"]/.cshrc[/font]:
Code: Select all
# set ccache varibles
setenv PATH /usr/local/libexec/ccache:$PATH
setenv CCACHE_PATH /usr/bin:/usr/local/bin
setenv CCACHE_DIR /var/tmp/ccache
setenv CCACHE_LOGFILE /var/log/ccache.log

# set ccache temp size to 512MB (default 1GB)
if ( -x /usr/local/bin/ccache ) then
  /usr/local/bin/ccache -M 512m > /dev/null
endif


logout and login again before using ccache because when You dont ccache will use [font="Courier New"]/root/.ccache[/font] dir instead of [font="Courier New"]/var/tmp/ccache[/font].

ccache can be shared between several computers the same as ports tree for example, check man ccache for more info.

ADDED 2007.2.15:
example ccache stats from my box.
Code: Select all
% ccache -s
cache directory                     /var/tmp/ccache
cache hit                          18562
cache miss                        102820
called for link                     9824
multiple source files                 75
compile failed                      1610
preprocessor error                  1446
not a C/C++ file                    3747
autoconf compile/link              16982
unsupported compiler option          511
no input file                       6698
files in cache                     49631
cache size                         464.3 Mbytes
max cache size                     512.0 Mbytes


ADDED 2007.2.16:
comparasion of buildworld times with and without ccache:
Code: Select all
# without ccache
make -j1 buildworld  4148.38s user 937.02s system 97% cpu [B]1:27:00[/B].40 total

# with ccache
make -j1 buildworld  1043.30s user 703.76s system 88% cpu [B]32:50[/B].02 total


If you face a problem with [font="Courier New"]/root/.ccache[/font] being used instead of [font="Courier New"]/var/tmp/ccache[/font] then you may solve it that way:

Code: Select all
% cd /home/${USER}
% rm -rf .ccache
% ln -s /var/tmp/ccache .ccache
% ls -l .ccache
lrwxr-xr-x  1 ${USER}  ${USER}  15 Dec 19 15:20 .ccache -> /var/tmp/ccache
% ccache -s
cache directory                     /var/tmp/ccache
cache hit                         165292
cache miss                        327142
called for link                    38002
multiple source files                216
compile failed                      5182
preprocessor error                  4934
couldn't find the compiler             1
not a C/C++ file                   26249
autoconf compile/link              52665
unsupported compiler option         1379
no input file                      23289
files in cache                     79438
cache size                         530.2 Mbytes
max cache size                     512.0 Mbytes
% sudo ccache -s
cache directory                     /home/vermaden/.ccache
cache hit                         165292
cache miss                        327142
called for link                    38002
multiple source files                216
compile failed                      5182
preprocessor error                  4934
couldn't find the compiler             1
not a C/C++ file                   26249
autoconf compile/link              52665
unsupported compiler option         1379
no input file                      23289
files in cache                     79438
cache size                         530.2 Mbytes
max cache size                     512.0 Mbytes
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby kamikaze » 17 Nov 2008, 08:19

<shameless advertising>
I'm using buildflags from sysutils/bsdadminscripts to do that. It also deals with using both, ccache and distcc at the same time.
</shameless advertising>

I have come to the conclusion that the default cache size of 1GB is way too low. At the moment I'm using 4GB:
Code: Select all
# ccache -s
cache directory                     /root/.ccache
cache hit                         607814
cache miss                        993734
called for link                    76913
multiple source files                316
compile failed                     11642
preprocessor error                  7718
couldn't find the compiler             4
not a C/C++ file                   90356
autoconf compile/link              97184
unsupported compiler option        14339
no input file                      29725
files in cache                    269281
cache size                           3.6 Gbytes
max cache size                       4.0 Gbytes


As you can see I have a hit rate of ~38%, which is a pretty impressive number, I think. I don't know how mature that previous example is (ccache has to be in use for some time to bring benefits), but the shown 512MB cache only has a hit rate of ~15%
User avatar
kamikaze
Member
 
Posts: 366
Joined: 17 Nov 2008, 07:34
Location: /earth/europe/germany

Postby vermaden » 17 Nov 2008, 08:43

kamikaze wrote:<shameless advertising>
I'm using buildflags from sysutils/bsdadminscripts to do that. It also deals with using both, ccache and distcc at the same time.
</shameless advertising>

I have come to the conclusion that the default cache size of 1GB is way too low. At the moment I'm using 4GB (...) As you can see I have a hit rate of ~38%, which is a pretty impressive number, I think. I don't know how mature that previous example is (ccache has to be in use for some time to bring benefits), but the shown 512MB cache only has a hit rate of ~15%


Thanks for tips, propably it was used for short period of time, the last stats from my post give these stats with 512MB size:
Code: Select all
cache hit                         165292
cache miss                        327142


About 33% so not so bad.
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby kamikaze » 17 Nov 2008, 09:10

More than 30% is very decent. Compiling OpenOffice would blow your cache apart, though. Everything useful would be flushed out. OpenOffice is the reason I stepped from 1GB to 4GB.
User avatar
kamikaze
Member
 
Posts: 366
Joined: 17 Nov 2008, 07:34
Location: /earth/europe/germany

Postby vermaden » 17 Nov 2008, 09:16

Compiling OpenOffice is like suicide, I always use packages for such big blobs like that ;)
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby kamikaze » 17 Nov 2008, 09:27

You normally have to wait for a long time for new OOo packages. That's why I build them myself and share them.
User avatar
kamikaze
Member
 
Posts: 366
Joined: 17 Nov 2008, 07:34
Location: /earth/europe/germany

Postby vermaden » 17 Nov 2008, 09:52

Propably I do not use OpenOffice that much to care about newest version, current 2.4 version seems to work fine.

Where do you keep your builds mate?

Maybe I will try them some day.
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby kamikaze » 17 Nov 2008, 10:05

http://wiki.bsdforen.de/anwendungen/openoffice_aus_inoffiziellen_paketen

I've switched over to 3, because it doesn't have the font rendering problems of the 2-branch.
User avatar
kamikaze
Member
 
Posts: 366
Joined: 17 Nov 2008, 07:34
Location: /earth/europe/germany

Postby Weaseal » 02 Jan 2009, 01:15

Is there an ad-hoc way to disable CCACHE? For example, something like:
Code: Select all
make install clean WITHOUT_CCACHE=yes
Weaseal
Junior Member
 
Posts: 57
Joined: 16 Nov 2008, 16:34

Postby vermaden » 02 Jan 2009, 01:44

like that mate:
[cmd=]# make NO_CACHE=yes install clean[/cmd]
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby kegf » 14 Jan 2009, 07:15

change
vermaden wrote:
Code: Select all
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CCX= /usr/local/libexec/ccache/world-cc++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif



to
Code: Select all
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CCX= /usr/local/libexec/ccache/world-c++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_CCACHE= yes
.endif
kegf
Junior Member
 
Posts: 12
Joined: 17 Nov 2008, 04:51

Postby vermaden » 14 Jan 2009, 07:48

@kegf

Thanks.
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby Mel_Flynn » 13 Feb 2009, 20:30

Add a note:
You have to reinstall devel/libtool15 if it's installed, as changing CC/CXX will screw things up for it.

Adding the PATH change is not required and can create problems with lang/gcc* ports as dependencies, as it will add gcc## to $PATH and BUILD_DEPENDS will think it's installed, but once compilation starts it doesn't work.

I use:
:setenv=....<stripped other stuff>,CCACHE_DIR=/var/db/ccache/$:

in /etc/login.conf to make CCACHE_DIR independant of shell semantics and dot.profile files.

Finally the stats from my package builder:
Code: Select all
cache directory                     /var/db/ccache/root
cache hit                         508615
cache miss                        379447
called for link                    66381
multiple source files                228
compile failed                     13214
ccache internal error                  2
preprocessor error                 21943
not a C/C++ file                   27067
autoconf compile/link             118453
unsupported compiler option         6337
no input file                      38147
files in cache                    650587
cache size                           8.6 Gbytes
max cache size                      10.0 Gbytes
Mel_Flynn
Member
 
Posts: 379
Joined: 18 Nov 2008, 18:02
Location: Drachten, Netherlands

Postby dennylin93 » 27 Jul 2009, 12:57

vermaden wrote:
Code: Select all
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CCX= /usr/local/libexec/ccache/world-c++
.endif



A small question. It is CCX or CXX? In [file]/usr/local/share/doc/ccache/ccache-howto-freebsd.txt[/file], is's CXX.
dennylin93
Member
 
Posts: 784
Joined: 11 Dec 2008, 13:13

Postby vermaden » 27 Jul 2009, 13:15

Its CXX.
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby Artefact2 » 27 Jul 2009, 22:38

Thanks for this tip.

A little precision : if you use a non-csh shell, you have to edit the rc file of your shell, not /.cshrc. The syntax might change (export instead of setenv, ...).
User avatar
Artefact2
Junior Member
 
Posts: 23
Joined: 29 Mar 2009, 20:43
Location: France

Postby vermaden » 28 Jul 2009, 06:01

@Artefact2

That's obvious, but thanks for pointing it out.
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby graudeejs » 22 Aug 2009, 19:17

My Stats:

I build OpenOffice.org-3.1 uncached (with ccache on) in about 10-11 hours

It took about 3h 40min to compile OOO3.1 when it was cached in ccache :)

My PC:
Pentium 4 HTT @ 3GHz
RAM: 2.5GB @ 400Mhz
on GELI encrypted zpool :)

Code: Select all
killasmurf86 $ ccache -s
cache directory                     /var/db/ccache
cache hit                          79143
cache miss                        139100
called for link                    14059
multiple source files                 75
compile failed                      3599
ccache internal error                  1
preprocessor error                  2494
couldn't find the compiler             2
not a C/C++ file                    7095
autoconf compile/link              28680
unsupported compiler option          932
no input file                      10850
files in cache                    278200
cache size                           2.6 Gbytes
max cache size                       4.0 Gbytes
User avatar
graudeejs
Style(9) Addict
 
Posts: 4591
Joined: 16 Nov 2008, 23:23
Location: Riga, Latvia

Postby nal » 26 Sep 2009, 20:38

Code: Select all
# Allow CCACHE for PORTS only (the buildworld has error with CCACHE)
.if ${.CURDIR:M*/ports*}
.if exists(/usr/local/libexec/ccache/world-cc) && !defined(NO_CCACHE)
CC=/usr/local/libexec/ccache/world-cc
CXX=/usr/local/libexec/ccache/world-c++
.endif
CFLAGS= ...
CXXFLAGS=${CFLAGS}
...
...
.endif
nal
Junior Member
 
Posts: 13
Joined: 16 Nov 2008, 18:31
Location: Russia, Orenburg

Postby Seeker » 12 Apr 2010, 21:41

vermaden wrote:like that mate:
[cmd=]# make NO_CACHE=yes install clean[/cmd]


A lots of luniz here will just copy-paste it and hit a wall:
Code: Select all
.if !defined(NO_CACHE)
  CC=  /usr/local/libexec/ccache/world-cc
  CXX= /usr/local/libexec/ccache/world-c++
.endif

.if ${.CURDIR:M*/ports/devel/ccache}
  NO_[color="Red"][B]C[/B][/color]CACHE= yes
.endif
User avatar
Seeker
Member
 
Posts: 855
Joined: 19 May 2009, 19:47
Location: Europe/Croatia/Zagreb

Postby vermaden » 12 Apr 2010, 21:50

@Seeker

Thanks, its also funny how long it was here without notifying ;)
Religions, worst damnation of mankind.
"FreeBSD has always been the operating system that GNU/Linux should have been." Frank Pohlmann, IBM
[FILE][color="DarkSlateBlue"]http://vermaden.blogspot.com[/color][/FILE]
User avatar
vermaden
Giant Locked
 
Posts: 2316
Joined: 16 Nov 2008, 19:37
Location: pl_PL.lodz

Postby mfaridi » 13 Jun 2010, 06:03

my apace of /var is limit and I use this command
Code: Select all
portmaster -D -r -w gettext
and I see ccache take many space of
Code: Select all
/var
what happen when I delete all files in ccache folder , because I do not have many space ?
User avatar
mfaridi
Member
 
Posts: 622
Joined: 17 Nov 2008, 05:32
Location: Afghanistan

Postby dennylin93 » 13 Jun 2010, 08:30

Simply limit the size of your cache using [cmd=#]ccache -M[/cmd]

If you delete the files, your cache will be empty.
dennylin93
Member
 
Posts: 784
Joined: 11 Dec 2008, 13:13

Postby Slade » 09 Aug 2010, 06:26

I'm wondering how I would go about using a new version of gcc (gcc45) with ccache. Right now I have just gcc45 running per the instructions at: http://www.freebsd.org/doc/en/articles/custom-gcc/article.html

That works fine but how would I go about getting ccache to use that. It seems like the instructions are for the standard version of gcc that comes with FreeBSD.

My current /etc/make.conf
Code: Select all
.if !empty(.CURDIR:M/usr/ports/*) && exists(/usr/local/bin/gcc45)
CC=gcc45
CXX=g++45
CPP=cpp45
CPUTYPE?=amd64
.endif


my current /etc/libmap.conf
Code: Select all
libgcc_s.so.1   gcc45/libgcc_s.so.1
libgomp.so.1    gcc45/libgomp.so.1
libobjc.so.3    gcc45/libobjc.so.2
libssp.so.0     gcc45/libssp.so.0
libstdc++.so.6  gcc45/libstdc++.so.6


I've tried looking over the examples in this thread and the ccache-howto-freebsd.txt when I installed ccache from ports but I just don't know enough about the make process to figure out how to get it working with gcc45.
Slade
Junior Member
 
Posts: 19
Joined: 23 Jul 2010, 00:36

Postby oliverh » 09 Aug 2010, 13:16

Using e.g. USE_GCC= 4.5+ is sufficient to my understanding.

According to http://ftp2.pl.freebsd.org/pub/FreeBSD/FreeBSD-current/ports/Mk/bsd.gcc.mk
What was the goal of the Linux community--to replace Windows? One can imagine higher aspirations., Bill Joy
User avatar
oliverh
Member
 
Posts: 557
Joined: 16 Nov 2008, 19:21
Location: 127.0.0.1

Next

Return to Howtos & FAQs (Moderated)

Who is online

Users browsing this forum: No registered users and 0 guests