1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

HOWTO: FreeBSD with CCACHE

Discussion in 'Howtos and FAQs (Moderated)' started by vermaden, Nov 16, 2008.

  1. camelia

    camelia New Member

    Messages:
    43
    Likes Received:
    0
    BTW, I'd suggest using `?=' everywhere. It leaves the chance to override value from environ(7) without touching make.conf.

    As for port's gcc there are few pitfalls but libmap.conf addresses only runtime linking problem completely avoiding build linking one (rtld vs. ld). BTW, using CPUTYPE?=native should work for any GCC (newer than gcc3*) and clang. If you're after more portable binaries then you'd want to use -mtune=generic but there is no make(1) macro for it.
     
  2. Seeker

    Seeker New Member

    Messages:
    856
    Likes Received:
    0
    I have 2 questions:

    1) Why exactly, do we have to recompile, devel/libtool22, after we change CC and CXX? What are the consequences, of not doing so?

    2) WITH ccache, I did a buildkernel and it took 43 min, to finish. Without ccache it took 39 min, to finish.
    Then mistakenly, I've issued a same command again and ONLY then it did compiled in 9 minutes!
    So, this means that ccache, speeds up RE-compilation, not a initial compilation, which can be even slower!
    Now, as far as I know for myself, after build, I go for install.
    I can't imagine, even one scenario, where I would go and build already build thing once again.
     
  3. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,180
    Likes Received:
    10
    Same as a browser cache. The first time you go to that page, it's a little slower because it has to cache everything. Second time, it only retrieves the things that have changed, so it's faster. Yes, there's some added overhead in saving the cached files, but it's usually faster than retrieving them again.

    When you buildworld or kernel, odds are good that a lot of it hasn't changed since the last time it was built, even weeks or months ago. How much has changed depends on how long it's been since you last built world. Next time you buildworld, time it again. If it takes less than 35 minutes (39 minutes normal minus the four already lost to overhead)... you win! After that, anything less than 39 minutes is pure win.
     
  4. Seeker

    Seeker New Member

    Messages:
    856
    Likes Received:
    0
    Well I did timed it here and after initial kernel compilation of 43 min, RE-compilation was done in 9 min.
    Which is almost, 5 times better, performance boost.

    Even few months?!
    In that time frame, I would do many, many compilations, of various ports, etc...
    Perhaps, a dedicated HDD, just for CCACHE_DIR /var/tmp/ccache, would suffice? :p
     
  5. vermaden

    vermaden Member

    Messages:
    2,352
    Likes Received:
    0
    @Seeker

    These are mine ccache stats after about 12 months of usage:
    Code:
    % zfs list basefs/var/ccache
    NAME                USED  AVAIL  REFER  MOUNTPOINT
    basefs/var/ccache   575M   569G   575M  /var/ccache
    
    % ccache -s
    cache directory                     /var/ccache
    cache hit                          17306
    cache miss                         43982
    called for link                     5148
    multiple source files                  5
    compile failed                      3107
    preprocessor error                   819
    not a C/C++ file                    3423
    autoconf compile/link               3957
    unsupported compiler option         1221
    no input file                       1343
    files in cache                     87964
    cache size                           1.1 Gbytes
    max cache size                       4.0 Gbytes
    Also, if You limit it to 1GB in config, then it will STAY 1GB size.
     
  6. ohauer

    ohauer New Member

    Messages:
    120
    Likes Received:
    0
    Hm, think about you maintain some ports and you will test all different build options
    or you patch parts of the source, generate the patchfiles and build again ...

    With usage of CCACHE the configure scripts are the main bottleneck for me, and I have similar values like vermaden.
     
  7. Nomacoro

    Nomacoro New Member

    Messages:
    1
    Likes Received:
    0
    i cant open the link. what is the matter?
     
  8. camelia

    camelia New Member

    Messages:
    43
    Likes Received:
    0
    Most of the time changing configure options affects config.h or CFLAGS that cause cache misses during build.
     
  9. Beeblebrox

    Beeblebrox Member

    Messages:
    868
    Likes Received:
    1
    ccache with GCC45

    I am using gcc45 for ports, native (gcc42) for source. Therefore the conditional statement in make.conf needs to be a little different for my setup. After some debugging, I seem to have found the correct settings for my make.conf
    Code:
    .if !empty{.CURDIR:M/usr/ports/*}
    CC=  /usr/local/libexec/ccache/world[color="Red"][B]/gcc45[/B][/color]
    CXX= /usr/local/libexec/ccache/world[color="red"][B]/g++45[/B][/color]
    CFLAGS+= -mssse3
    .else 
    CC=  /usr/local/libexec/ccache/world[color="Red"][B]/cc[/B][/color]
    CXX= /usr/local/libexec/ccache/world[color="red"][B]/c++[/B][/color]
    .endif
    
    .if ${.CURDIR:M*/ports/devel/ccache}
      NO_CCACHE= yes
      CC= gcc45
      CXX= g++45
      CPP= cpp45
      CFLAGS+= -mssse3
    .endif
    Check your /usr/local/libexec/ccache/world/ directory and make sure you have all of the above named files in the folder. These of course, are not files but soft-links to the ccache file in the same folder. If you do not have such links, create them (#ln -s) with the appropriate names specified here (ccache should have created them already though).

    Whenever you change (upgrade) ccache versions or add a different compiler (say you switched form 4.5 to 4.6 or such) you must re-build devel/libtool, otherwise you will see "precompiler fails sanity check" messages.
     
  10. qsecofr

    qsecofr New Member

    Messages:
    222
    Likes Received:
    0
    I needed to edit /etc/make.conf as suggested by Beeblebrox:
    Code:
    CC=  /usr/local/libexec/ccache/world[B][color="Red"]/[/color][/B]cc
      CXX= /usr/local/libexec/ccache/world[B][color="Red"]/[/color][/B]c++
    The original suggestion had dashes instead of forward slashes. Not sure if the original was a typo or worked as intended on author's system. I'm using FreeBSD 8.1-Release.

    Also replies suggested building libtool15. I have a more recent version. Not sure if that needs rebuilding, but I did. Was there anything version-specific about that recommendation?

    It would have been good if the HOWTO referenced /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and explained any differences.

    I agree with Seeker in that a distinction exists between compilation vs. re-compilation. I have no idea how many times I re-compile the same version of the same port. Not by intent anyway - I leave that decision up to portmaster and or make. But I'm hopeful the tool has finer granularity than just overall port version. I'll probably know the next time I upgrade KDE. Maybe.
     
  11. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,180
    Likes Received:
    10
    Benchmark it. Set NO_CCACHE and time a build. Reboot to clear the filesystem cache, leave ccache enabled, and build it again. My guess for port updates is that ccache usually won't have it in the cache and will actually make port builds a little slower.

    /usr/src doesn't change all that much from week to week, but there can still be a lot of cache misses caused by small changes.
     
  12. vermaden

    vermaden Member

    Messages:
    2,352
    Likes Received:
    0
    CCACHE is useful if you compile something at least TWO TIMES; the first time it's slower because it 'learns', the second and every other compile run is a lot faster. Not the first one.
     
  13. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,180
    Likes Received:
    10
    Of course. But do you think the cached version of KDE will still be in the cache by the time the next version comes out? And how much of it will be valid because the source hasn't changed? If it's not in the cache or invalid, the overhead of ccache will slow you down, and I suspect that will be the case most of the time with ports. Maybe a really big cache size would help. ccache -s will tell the story.
     
  14. vermaden

    vermaden Member

    Messages:
    2,352
    Likes Received:
    0
    I would say that if you compile a lot and have a quite large ccache space (4-8G) then the more you compile, the faster it will be, at least I got that impression after longer usage. Currently I only use packages and compile as little as possible, so I do not even bother with setting up CCACHE anymore.
     
  15. bbzz

    bbzz New Member

    Messages:
    867
    Likes Received:
    0
    I have a habit of compiling everything as soon as it's updated. When I'm building as root it works. When I'm building as user using sudo it never uses CCACHE.

    The configuration for root is in ~/.cshrc. The configuration for user is in ~/.zshrc. The configuration is identical, minus shell semantics, and both point to same CCACHE location, /var/db/ccache. So the both $PATH are equal.

    Since I use sudo with user to update ports and build, CCACHE is never used. What do I need to do to make this work?

    p.s. great guide as always.
     
  16. vermaden

    vermaden Member

    Messages:
    2,352
    Likes Received:
    0
    Edit the /usr/local/etc/sudoers file and add needed variables that should be kept when using sudo(8), for example:
    Code:
    Defaults        env_keep += "HOME PKG_PATH PKG_DBDIR PKG_TMPDIR TMPDIR PACKAGEROOT PACKAGESITE PACKAGES PKGDIR FTP_PASSIVE_MODE"
     
  17. bbzz

    bbzz New Member

    Messages:
    867
    Likes Received:
    0
    Tried, but that doesn't help.
    There is no update when I check cache with:
    # ccache -s
    Whereas, when run as root it get updated.

    EDIT: I just realized that when using sudo user uses /root/.ccache.
    EDIT2: Hence, fixed! Thx.
     
  18. graudeejs

    graudeejs Member

    Messages:
    4,594
    Likes Received:
    0
    That's because If you use shared ccache you'll probably have permissions issues
     
  19. bbzz

    bbzz New Member

    Messages:
    867
    Likes Received:
    0
    I changed permissions there. For some reason it only works when configuration is defined in both my user's ~/.zshrc AND settings forwarded in visudo.
    If it's only configured in ~/.zshrc, but not in visudo (CCACHE_PATH, CCACHE_DIR), it uses default /root/.ccache location. If it's only defined in root's ~/.cshrc and forwarded with visudo, (so nothing defined in user's ~/.zshrc), then it uses user's default location, ~/home/$user/.ccache.
    I don't get this, but I got it to work.
     
  20. graudeejs

    graudeejs Member

    Messages:
    4,594
    Likes Received:
    0
    This should be updated to

    Code:
    .if !defined(NO_CCACHE)
      CC=  /usr/local/libexec/ccache/world/cc
      CXX= /usr/local/libexec/ccache/world/c++
    .endif
    
     
  21. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,180
    Likes Received:
    10
    The port suggests
    Code:
    .if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
    .if !defined(NOCCACHE)
    CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
    CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
    .endif
    .endif
    
     
  22. graudeejs

    graudeejs Member

    Messages:
    4,594
    Likes Received:
    0
    Yes, btw, I use
    Code:
    WRKDIRPREFIX=/usr/obj
    When I used
    Code:
    if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj/usr/src*))
    ...
    .endif
    
    it (ccache) didn't work. Any idea why?
     
  23. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,180
    Likes Received:
    10
    That line uses ccache only for building the system. That's pretty reasonable, it's rare that ccache will help with building ports.
     
  24. bbzz

    bbzz New Member

    Messages:
    867
    Likes Received:
    0
    Code:
    .if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj/usr/src/*))
    .if !defined(NOCCACHE)
    CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
    CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
    .endif
    .endif
    
    So this is the correct way to specify ccache only for world building and not ports? Is there an optimum size for ccache maybe?
     
  25. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,180
    Likes Received:
    10
    I've used sizes of 1G to 4G, and more really doesn't help. Either nothing major has changed in the source and it mostly comes from cache, or something major or minor has changed that invalidates all of the cache, and everything has to be rebuilt.