1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[HOWTO] buildworld using clang + ccache

Discussion in 'Howtos and FAQs (Moderated)' started by Beeblebrox, Nov 23, 2012.

  1. Beeblebrox

    Beeblebrox Member

    Messages:
    894
    Thanks Received:
    110
    clang is now the default compiler for FreeBSD (instead of gcc-42), and things supposedly run quicker under clang. You can do the buildworld with clang, but including ccache in the process needs some small adjustments:

    1. Edit your /etc/src.conf as below. clang is enabled by default anyway, but you may want to add extras to it. Originally this post advised WITHOUT_GCC for a quicker build, but this is ill-advised as explained below.
    Code:
    WITH_CLANG= yes
    WITH_CLANG_EXTRAS= yes # Build additional clang and llvm tools, such as bugpoint.
    WITH_CLANG_IS_CC= yes  # Useful for some buildworld errors
    # WITHOUT_GCC= yes


    2. In your /usr/local/etc/buildflags.conf set:
    Code:
    /usr/src | /usr/src/*{
            CC=     clang
            CPP=    clang-cpp
            CXX=    clang++
            USE_CCACHE
            USE_CCACHE_CPP2
    }


    You are now set to buildworld with clang + ccache. CCACHE_CPP2 must be enabled because clang dislikes ccache trying extra flags.

    If you do not use sysutils/bsdadminscripts, and therefore do not have buildflags.conf, you can set those parameters from command line :
    [CMD="/usr/src> #"]make -DUSE_CCACHE -DUSE_CCACHE_CPP2 buildworld[/CMD]
    In this situation you of course need these in your /etc/make.conf.
    Code:
    CC=     clang
    CPP=    clang-cpp
    CXX=    clang++


    Thanks to Volodymyr Kostyrko for helping me solve this.

    EDIT: Apparently as of the date of this post, setting WITHOUT_GCC= yes does not work yet and breaks buildworld. It must be left in /etc/src.conf for now.
     
  2. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,550
    Thanks Received:
    2,223
    What ccache environment variables or /etc/make.conf settings are used with this?

    Although it will increase compile time, it might be a good idea to leave gcc enabled in src.conf just to have it available.
     
    graudeejs thanks for this.
  3. Beeblebrox

    Beeblebrox Member

    Messages:
    894
    Thanks Received:
    110
    No additional ccache environemnt variables / settings required than when using gcc-4* (CCACHE_PATH, CCACHE_DIR) and setting the size (# ccache -M <size>)

    The config file for building ccache now has the options for
    Code:
    CLANGLINK
    LLVMLINK

    These, when selected will automagically create FILE]ccache[/FILE] shortcuts for clang to ccache.

    As a Third way of doing things, specifying these in /etc/make.conf should work:
    Code:
    CC=/usr/local/libexec/ccache/world/clang
    CXX=/usr/local/libexec/ccache/world/clang++
    CPP=/usr/local/libexec/ccache/world/clang-cpp


    I use lang/gcc46 for compiling my ports. Users who do not have a separate gcc should not disable gcc42 because a good number of ports still require gnu-gcc to compile. Invoke WITHOUT_GCC= yes only if you plan to use a higher version of gcc for your ports.
     
  4. xeube

    xeube New Member

    Messages:
    15
    Thanks Received:
    4
    Did I miss something? I tought that clang was to become the default compiler starting with 10.0.
     
  5. kpa

    kpa Well-Known Member

    Messages:
    4,083
    Thanks Received:
    804
    It will become the default compiler in CURRENT but it's not yet done I believe.

    It's a good idea to leave GCC in base because of some potential hickups in buildworld. Just recently it was impossible to compile parts of the boot code with clang(1) for a while and if you don't have GCC as the fallback compiler you can't finish the build until the problem is fixed in SVN.
     
  6. Beeblebrox

    Beeblebrox Member

    Messages:
    894
    Thanks Received:
    110
    @xeube: Yes, clang is default for HEAD only. Its announcement as default is recent. Before that, it was only targeted / planned as default.

    @kpa: The ride is certainly far from smooth...
     
  7. ColdfireMC

    ColdfireMC Member

    Messages:
    140
    Thanks Received:
    6
    does CLANG have advantages compared with base gcc?
     
    ShelLuser thanks for this.
  8. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,550
    Thanks Received:
    2,223
    Supposed to compile faster and have better error messages. Also a BSD-compatible license.
     
    ShelLuser thanks for this.
  9. Beeblebrox

    Beeblebrox Member

    Messages:
    894
    Thanks Received:
    110
    UPDATE: I just finished buildworld with settings:
    Code:
    WITHOUT_GCC= yes
    WITH_CLANG= yes
    WITH_CLANG_EXTRAS= yes
    WITH_CLANG_IS_CC= yes

    However, ccache does not always play well with clang and needs to be disabled if build stops.
    This means, start with ccache enabled, but if build stops, disable ccache and <threads> then re-start, OK?

    @ ColdfireMC: Olivier Smedts informed me of below - your call:
     
    ShelLuser thanks for this.
  10. ColdfireMC

    ColdfireMC Member

    Messages:
    140
    Thanks Received:
    6
    but...? are there problems about that?

    I recently read that binaries are slightly faster, because of newer and better cpu support than gcc4.2 but is near the same versus newer(or not-so-newer) gcc versions

    is that true?

    well, stop depending on a GNU (or another license with compatiblity issues with BSD license and philosophy) is a huge step towards gaining community independence, keep it up :).
     
  11. kpa

    kpa Well-Known Member

    Messages:
    4,083
    Thanks Received:
    804
    GCC has been around lot longer than clang and it may perform slightly better code optimizations in some cases even on an old version like 4.2.1.
     
    ShelLuser and jotawski thanked for this.
  12. Beeblebrox

    Beeblebrox Member

    Messages:
    894
    Thanks Received:
    110
    UPDATE: I built my clang-only world and kernel with:
    Code:
    WITH_CLANG= yes
    WITH_CLANG_EXTRAS= yes
    WITH_CLANG_IS_CC= yes
    WITHOUT_GCC= yes

    CLANG_IS_CC is definitely needed if you decide to apply WITHOUT_GCC. I was however, forced to disable ccache at some point so as to continue the build from the point that buildworld broke (using -DNO_CLEAN). Then the build completed without any other problems. Running my clang-only world now :)
     
  13. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Sorry for asking, but does clang natively include ccache, or should one install it separately from port devel/ccache?
     
  14. kpa

    kpa Well-Known Member

    Messages:
    4,083
    Thanks Received:
    804
    No, it's a separate tool and supports other compilers as well. You have to install the port devel/ccache
     
    MNIHKLOM and jozze thanked for this.
  15. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Cool, thanks for the info ;).
     
  16. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,550
    Thanks Received:
    2,223
    Last time I tested, ccache really did not make much of an improvement to clang buildworld times. If anyone wants to benchmark, clear the cache ( ccache -C) and run two timed buildworlds. The first should take about as long as without ccache, the second should show the improvement in speed.

    Incidentally, setting CCACHE_COMPRESS=yes seems to have no speed penalties and fits more in the cache.
     
    ShelLuser and MNIHKLOM thanked for this.
  17. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Where are all these variables defined? I have been trying to find them in /usr/src, but they're not there.
     
  18. kpa

    kpa Well-Known Member

    Messages:
    4,083
    Thanks Received:
    804
    In the devel/ccache documentation. It doesn't make any sense to make the system sources and the build tools dependent on an external tool, that's why you won't see any mention of devel/ccache in the system sources or the build(7) manual page.
     
    jozze thanks for this.
  19. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Pardon my ignorance, but $ man ccache and files in /usr/share/doc/ccache folder describe no such variables as USE_CCACHE or USE_CCACHE_CPP2, which is why I am asking. There is CCACHE_CPP2, however. I am not sure, if the approach described in this how-to gives the expected results, because the /usr/local/share/doc/ccache/ccache-howto-freebsd.txt suggests a different way, to use devel/ccache:

    Code:
    #
    # $FreeBSD: head/devel/ccache/files/ccache-howto-freebsd.txt.in 300896 2012-07-14 13:54:48Z beat $
    #
    
    To use ccache add the following to /etc/make.conf.
    You can replace cc and c++ with the compilers of your choice.
    (remember that only GCC and Clang can build world and kernel)
    
    .if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
    .if !defined(NOCCACHE)
    CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
    CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
    .endif
    .endif
    
    For Korn/Bourne shells Add the following to /etc/profile:
     export PATH=/usr/local/libexec/ccache:$PATH
     export CCACHE_PATH=/usr/bin:/usr/local/bin
    
    For csh/tcsh Add the following to /etc/csh.cshrc:
     setenv PATH /usr/local/libexec/ccache:$PATH
     setenv CCACHE_PATH /usr/bin:/usr/local/bin
    
    For icc users:
     Add /usr/local/intel_cc_80/bin to CCACHE_PATH
    
    --
    
    To use distcc:
    
    For Korn/Bourne shells Add the following to /etc/profile:
     export CCACHE_PREFIX=distcc
     export DISTCC_HOSTS="localhost host1 host2"
    
    For csh/tcsh Add the following to /etc/csh.cshrc:
     setenv CCACHE_PREFIX distcc
     setenv DISTCC_HOSTS "localhost host1 host2"
    
    --
    
    If you have a problem building world
    define NOCCACHE and try again.
    
    If you have a problem building a port
    reset PATH=$CCACHE_PATH and try again.
    ...
    


    Or am I missing something here?
     
  20. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Yes, that's what I thought, because I didn't see any difference. So now, I made two consecutive buildworlds, but the impact was really great! I don't have sysutils/bsdadminscripts installed, I didn't set the USE_CCACHE or USE_CCACHE_CPP2 variables. What I did is described below.

    I installed devel/ccache with these config flags enabled:

    Code:
    # make -C /usr/ports/devel/ccache pretty-print-config
    +CLANGLINK +DOCS +LLVMLINK -STATIC
    


    My make.conf and my src.conf. Notice that I don't have CCACHE_CPP2 enabled. After that I added these two variables as the /usr/local/share/ccache/ccache-freebsd.txt suggested:

    Code:
    # echo "setenv CCACHE_PATH /usr/bin:/usr/local/bin" >> ~/.tcshrc
    # echo "setenv CCACHE_LOGFILE ~/.ccache.log" >> ~/.tcshrc
    # echo "setenv CCACHE_COMPRESS 1" >> ~/.tcshrc
    


    I decided to keep a log, if something goes wrong, and to confirm that I am really using ccache to do the job, and to make compression (it really paid off -- from ~600 MB it reduced cache size to 80 KB).

    After that I ran
    Code:
    # cd /usr/src
    # time make -j16 buildworld
    # time make -j16 buildworld
    


    The first time it took me ~40 min to build (from 22:24 to 22:04). The second time it took considerably less: ~13 minutes (from 22:14 to 22:27).
     
  21. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,550
    Thanks Received:
    2,223
    I use a slightly different entry in make.conf:
    Code:
    .if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*)) && !defined(NOCCACHE)
      CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
      CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
    .endif
    


    The idea being to avoid the overhead of the extra "if" by short-circuiting. I have not tested whether that's actually an improvement.
     
    jozze thanks for this.
  22. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Thank you, that definitely makes it more readable, but I don't think it has an impact on performance, since I think this file is read only once on the beginning (by accident I once changed the compiler in it during compilation and nothing changed until the compilation ended).

    UPDATE:
    I noticed, after playing with devel/ccache that world can easily be built without CCACHE_CPP2 option, but the kernel will fail (fun fact).

    But what does the devel/ccache actually do? I read through the manual but I don't quite understand how does saving time or binary content speed up the compiling process. Does it reuse old binaries?
     
  23. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,550
    Thanks Received:
    2,223
    From http://ccache.samba.org/:
     
  24. kpa

    kpa Well-Known Member

    Messages:
    4,083
    Thanks Received:
    804
    "Compilation" in this context means only the pre-processor macro definitions that are used abundantly in C. Caching them provides quite significant savings in compilation times because the definitions can be read from a hash table in nearly constant time.
     
    jozze thanks for this.
  25. jozze

    jozze New Member

    Messages:
    340
    Thanks Received:
    66
    Oh, thanks @kpa! That's what was puzzling me, I thought that it was saving compilation results.
     
    Last edited by a moderator: Oct 16, 2014