1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

what's in your make.conf?

Discussion in 'Installing and Upgrading FreeBSD' started by nbittech, Dec 5, 2012.

  1. nbittech

    nbittech New Member

    Messages:
    31
    Thanks Received:
    3
    I originally moved to FreeBSD from Gentoo. And on Gentoo, make.conf is a very powerful thing. But will the Gentoo philosophy of optimization have as good of an effect with FreeBSD, or will it just break more things?

    I set a system up this way with noticeable results, but have yet to test every scenario.

    Does anyone even use custom cflags? Or does it lead to long compile times and little else?

    Will things like NO_PROFILE=YES and static libraries make a difference on slower systems?

    What are some general make.conf tuning suggestions?

    Does anything I did actually work? Or do I have it all wrong?

    What other tricks do people use? WITHOUT_NLS, setting KDE4_PREFIX to /usr/local, using knobs, etc.

    Maybe post some make.confs? People?

    Personaly, my make.conf on one system looks like this:

    Code:
    CPUTYPE=prescott (P4 3.6ghz w/ht)
    CFLAGS=-O2 -pipe -fno-strict-aliasing -fomit-frame-pointer (What? no -funroll-loops? Uh?)
    COPTFLAGS=-O2 -pipe -fno-strict-aliasing (remember, nothing above -O2 is supported!)
    CXXFLAGS+=-fconserve-space (adding when availible, as I have a small, 36GB hard disk)
    KERNCONF=NBIT
    MAKE_JOBS_NUMBER=3 (prescott w/hyperthreading)
    BUILD_OPTIMIZED=YES
    WITH_CPUFLAGS=YES
    BUILD_STATIC=YES (have fast 15k SCSI hdd)
    NO_PROFILE=YES
    OPTIMZED_CFLAGS=YES
    WITH_OPTIMIZED_CFLAGS=YES
    WITHOUT_NLS=YES (I live in the U.S.)
    WITH_X11=YES
    WITH_NEW_XORG=YES
    WITH_PKGNG=YES (of course)
    QT4_OPTIONS= CUPS QGTKSTYLE (makes QT4 apps look good in Xfce)
    KDE4_PREFIX=/usr/local (makes all QT4 apps go where they are supposed too)
     
  2. Uniballer

    Uniballer Member

    Messages:
    252
    Thanks Received:
    26
    This breaks nothing:
    Code:
    # added by use.perl 2012-11-27 08:06:31
    PERL_VERSION=5.14.2
    
     
  3. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,723
    Thanks Received:
    2,271
    Not even that good. Not only does it cause breakage, it prevents ports that can use optimized CFLAGS from using them (because you've forced the CFLAGS, overridden them). So it can make things slower.

    Maybe. Or maybe just smaller binaries.

    Setting CPUTYPE is usually okay. Otherwise, it's all speed holes and red paint.

    WITHOUT_X11=yes on servers.

    Use devel/ccache if you update source often, or rebuild ports often.
     
  4. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,723
    Thanks Received:
    2,271
    Things in my /etc/make.conf:
    Code:
    DIALOG="/usr/ports/Tools/scripts/dialogwrapper.sh"   # see http://forums.freebsd.org/showthread.php?t=34536
    CPUTYPE?=core2     # for i5
    WITH_NEW_XORG=yes
    WITHOUT_CUPS=yes
    RANDOMIZE_MASTER_SITES=yes
    MAKE_JOBS_NUMBER?=8
    RUBY_DEFAULT_VER=1.9   # to use ruby 1.9
    
     
  5. Beeblebrox

    Beeblebrox Member

    Messages:
    908
    Thanks Received:
    113
    I prefer to use sysutils/bsdadminscripts for passing different flags for world vs. ports build and on a per-port basis.
    Among other things, this port places buildflags.conf(1) which you can tweak to your heart's desire. Just the /usr/src section of that file for example:
    Code:
     /usr/src | /usr/src/*{
            CC=     clang
            CPP=    clang-cpp
            CXX=    clang++
            USE_CCACHE
            USE_CCACHE_CPP2
    #       USE_DISTCC
            THREADS=        6
            NO_CLEAN
    }

    Whereas for the ports section the list goes on-and-on, but also has this:
    Code:
            BUILDFLAGS_GCC=         4.6+
            GCC_DEFAULT_VERSION=    4.6+

    So I am able to use clang for world but gcc4.6 for ports.
     
  6. xibo

    xibo Member

    Messages:
    384
    Thanks Received:
    72
    The base system creates extra versions of all libraries suffixed with _p in their name for profiled code and those are being linked against if and only if an application was built with profiling enabled. Therefore their presence shouldn't slow down the system.

    Static linking would usually slow down applications though, as the binaries need to be paged in individually, while most shared libraries will already have been paged in by other applications that also use them if they also use dynamic linking.
     
  7. Toast

    Toast New Member

    Messages:
    49
    Thanks Received:
    5
    Nothing special.
    Code:
    VIDEO_DRIVER=ati
    WITH_LCD_FILTERING=1
    
    OPTIONS_UNSET= CUPS NLS DOCS
    WITHOUT_CUPS=1
    WITHOUT_NLS=1
    NOPORTDOCS=1
     
  8. bbzz

    bbzz Member

    Messages:
    868
    Thanks Received:
    123
    Even CPUTYPE shouldn't be set if you are building all your packages on one machine for all other. Also, setting CPUTYPE to native proved to be safest in the past.

    Code:
    WITHOUT_DEBUG=          YES
    NO_PROFILE=             YES
    BUILD_STATIC=           YES
    NO_NIS=                 YES
    WITH_LCD_FILTERING=     YES
    WITH_VDPAU=             YES
    WITHOUT_NOUVEAU=        YES
    WITHOUT_PULSEAUDIO=     YES
    WITH_PKGNG=             YES
    
    .if (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
    .if !defined(NOCCACHE)
    CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
    CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
    
     
  9. throAU

    throAU New Member

    Messages:
    912
    Thanks Received:
    134
    About the only option I tweak is WITHOUT_X11.

    Messing with random optimization flags just means that any bugs you encounter are likely to be very difficult to diagnose and/or reproduce by others.

    The small gains you'll get aren't worth it, IMHO.
     
  10. nbittech

    nbittech New Member

    Messages:
    31
    Thanks Received:
    3
    I will probably leave the cflags alone, thanks.

    I didn't realize that you can set your video card in make.conf.

    As for the famous "-O3 -funroll-loops -ffast-math" it will just lead to massive sized binaries that take forever to compile, and longer to load once compiled, and unless I have an large SSD and 8GB of RAM, (at that point who needs the performance anyway?) I see no benefit. Just not worth it to me.

    FreeBSD is wonderful because the ports system gives me the ability to experiment with stuff like this. You can do almost anything with it, it's like having a Swiss-Army-Knife of software.

    Oh, and one more question, clang or gcc? I've heard that clang provides faster compilation, whereas gcc produces better optimized code, but with longer compile times.
     
  11. throAU

    throAU New Member

    Messages:
    912
    Thanks Received:
    134
    Unless you're willing to be a tester and submit bug reports, use the default compiler for your release, as that is the one that has been deemed production ready.

    FreeBSD 10 is going to default to clang I believe, but in previous releases I believe it is still experimental and a work in progress.
     
  12. alie

    alie Member

    Messages:
    576
    Thanks Received:
    50
    Nothing special:

    Code:
    # added by use.perl 2012-11-19 16:19:19
    PERL_VERSION=5.16.2
    WITH_NEW_XORG=true
    WITH_KMS=true
    CC= /usr/local/bin/gcc47
    CXX= /usr/local/bin/g++47
    
    CFLAGS+=        -O3 -fno-strict-aliasing -pipe -funroll-loops
    CXXFLAGS+=      -O3 -fno-strict-aliasing -pipe -funroll-loops
    COPTFLAGS+=     -O3 -pipe -ffast-math -funroll-loops
     
  13. eadler@

    eadler@ New Member Developer

    Messages:
    24
    Thanks Received:
    14
    I think you mean "very special and very broken"

    Code:
    CFLAGS+=        -O3
    


    O3 adds the following optimizations:

    -finline-functions - this option considers every function for inlining even if it isn't specified with "inline". This breaks debuggers and can easily slow down your code (large code size implies more icache misses, less good branch prediction)

    -funswitch-loops
    A tradeoff of fewer branches vs larger code size.

    There is no guarantee of faster generated code.

    Code:
    -fno-strict-aliasing
    

    This actually makes the resulting code slower. This option exists for C programs which don't follow C's type aliasing rules.

    Code:
    -funroll-loops

    A tradeoff of fewer branches vs larger code size.

    Code:
    -ffast-math


    This option should be renamed -fbroken-math. Some programs or libraries depend on this option not being set. It breaks the math operations in C and C++ and completely disregards the standards: it exists for programs which do not care about the results of the math they compute.
     
    phoenix, throAU, dclau and 2 others thank for this.
  14. UNIXgod

    UNIXgod New Member

    Messages:
    1,089
    Thanks Received:
    195
    I use --omg-optimized. It's the fast sauce the make genttoo go zoooooom.

    http://funroll-loops.info

    Seriously though. As much as gentoo makes working with linux tolerable( and Funtoo takes that to the next step) even they recommend not using some of these flags: http://www.gentoo.org/doc/en/gcc-optimization.xml

    From their FAQ:
    I enjoy working with Funtoo on my laptop to hack around for one of two reasons. The hacker who put the meta-distribution together worked with FreeBSD for many years before he made his own GNU/Linux. It's educational as it truly is somewhere from sort of a custom linux-from-scratch through a semi-automated install. Finally it's probably the only GNU/Linux distro with FreeBSD derived concepts (i.e. portage) so it doesn't seem completely alien. The -zomg-fast-sauce
     
    nbittech thanks for this.
  15. nbittech

    nbittech New Member

    Messages:
    31
    Thanks Received:
    3
    I have to gree with UNIXGOD, optimized to the point of failure is pointless, as most people want a system that works, rather than bragging rights. You could build a car around an 18 cylinder locomotive engine, but it won't be very fast, just really large, heavy, and fuel thirsty. You probably wouldn't even be able to drive it. But build a small car with a turbocharged 4 cylinder engine, and it works much better. I get the point.

    When in doubt, leave it alone. Modern hardware is generally fast enough that I shouldn't need speed over stability. Less is more.
    I'm not doubting that there are some optimizations that do work, but the gains are so small that I most likely wouldn't even notice them anyway.
    I built a system entirely from packages, then rebuilt it all with optimized code once, and I couldn't even tell the difference, and this was a server that was under at least 70% load all of the time.

    The more things one messes with, the harder it becomes to solve problems.

    And I know that this is the FreeBSD forum, but Gentoo is fun as well. (I just prefer FreeBSD because it is so much simpler and less time-consuming.)
     
  16. kpa

    kpa Well-Known Member

    Messages:
    4,109
    Thanks Received:
    812
    Optimization trough compilers is the wrong approach anyway. You should be looking at more time efficient algorithms to improve your software if lower running time is the goal, not some magical compiler flags that supposedly can fix your naively written code automagically.
     
  17. nbittech

    nbittech New Member

    Messages:
    31
    Thanks Received:
    3
    If the code is crap, aggressive compiler flags will just make it worse. If the code is clean, and well-written, you shouldn't need extra flags anyway. There is no "magic bullet."

    But sometimes it is fun to see what happens!

    That's the point of hacking, right?

    When you have old hardware laying around (or nowdays, extra RAM and disk space for more VMs) it's easy to test things like this.

    I think that I should focus my time more on how I set up a system, rather than how I optimize the code.
     
  18. UNIXgod

    UNIXgod New Member

    Messages:
    1,089
    Thanks Received:
    195
    It's a better reason than some that I've heard. I imagine it also opens up some interest for new users to see how to mess with the compiler a bit even if they aren't c programmers. Not sure how much of these flags will be in clang, or is it llvm.

    On my laptop I have funtoo installed. I can run this and it dumps all the processor specific flags which are detected with --march=native. Mind you this is gcc version 4.6.3:

    % echo "" | gcc -march=native -v -E - 2>&1 | grep cc1
    Code:
     /usr/libexec/gcc/x86_64-pc-linux-gnu/4.6.3/cc1 -E -quiet -v - 
    -march=corei7-avx -mcx16 -msahf -mno-movbe -maes -mpclmul -mpopcnt -mno-abm 
    -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-tbm -mavx -msse4.2 
    -msse4.1 --param l1-cache-size=32 --param l1-cache-line-size=64 --param 
    l2-cache-size=4096 -mtune=corei7-avx


    with that I can just use native or copy and paste the results as so:
    Code:
    CFLAGS="-march=corei7-avx -mcx16 -msahf -mno-movbe -maes -mpclmul -mpopcnt 
    -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-tbm -mavx 
    -msse4.2 -msse4.1 --param l1-cache-size=32 --param l1-cache-line-size=64 
    --param l2-cache-size=4096 -mtune=corei7-avx -O2 -pipe"
    CXXFLAGS=${CFLAGS}


    With 9.1 on the horizon with intel video support I look forward to popping FreeBSD on my laptop. When I do I'll report back to this thread on any customizations worth investigating.

    nbittech do you program? When you start setting up your system try building yourself some custom automation scripts. If you want a interesting project to try. Create a script which deploys jails in one step. Experiment with different tunings and settings within your jails. Write a mini bench-marker to provide results on your experiments.

    Nothing wrong with hacking around as it's a great way to learn and explore the system. I used to mess with the FreeBSD make.conf() settings. The results won't be as over the top as you may have seen on your gentoo boxes.

    Another area which may interest you is src.conf().
     
  19. xtaz

    xtaz Member

    Messages:
    167
    Thanks Received:
    29
    I've always been sensible as regards optimization and assumed that the developers know best. I would rather have stable, known code, that I can debug easily. Instead I try and optimize the compilation rather than the run speed by using ccache and parallel make jobs and I've been experimenting with clang. I also have some logic for per-port build flags. My make.conf is pretty long but might give some interesting insights.

    Code:
    KERNCONF=TAO
    BATCH_DELETE_OLD_FILES=yes
    SVN_UPDATE=yes
    SVN=/usr/local/bin/svn
    #PORTSNAP_UPDATE=yes
    #SUP_UPDATE=yes
    #SUPFLAGS=-4 -g -L2
    #SUPFILE=/usr/local/etc/csup-src
    WRKDIRPREFIX=/usr/obj
    FORCE_PKG_REGISTER=yes
    DISABLE_VULNERABILITIES=yes
    WITHOUT_X11=yes
    
    .if ${.CURDIR:M*/shells/bash*}
      WITH_STATIC_BASH=yes
    .endif
    
    .if ${.CURDIR:M*/databases/mariadb-*}
      WITH_CHARSET=utf8
      WITH_COLLATION=utf8_unicode_ci
    .endif
    
    #.if ${.CURDIR:M*/www/apache*}
    #  WITH_OPENSSL_PORT=yes
    #.endif
    
    FORCE_MAKE_JOBS=yes
    MAKE_JOBS_NUMBER=4
    #.if ${.CURDIR:M*/multimedia/mplayer}
    #  .undef FORCE_MAKE_JOBS
    #.endif
    
    .if ${.CURDIR:M*/devel/ccache}
      NOCCACHE=yes
    .endif
    
    .if !defined(NOCCACHE)
    .if ${.CURDIR:M*/usr/src/*} || ${.CURDIR:M*/usr/obj/usr/src/*}
      CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/cc,1}
      CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/c++,1}
    #  CC:=${CC:C,^cc,/usr/local/libexec/ccache/world/clang,1}
    #  CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/world/clang++,1}
    #  CPP:=${CPP:C,^cpp,/usr/local/libexec/ccache/world/clang -E,1}
    .else
      CC:=${CC:C,^cc,/usr/local/libexec/ccache/cc,1}
      CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/c++,1}
    #  CC:=${CC:C,^cc,/usr/local/libexec/ccache/clang,1}
    #  CXX:=${CXX:C,^c\+\+,/usr/local/libexec/ccache/clang++,1}
    #  CPP:=${CPP:C,^cpp,/usr/local/libexec/ccache/clang -E,1}
    .endif
    .endif
    
    #CC=clang
    #CXX=clang++
    #CPP=clang-cpp
    
    .if ${CC:T} == "clang"
      CFLAGS+= -Qunused-arguments -fcolor-diagnostics
      NO_WERROR=
      WERROR=
    .endif
    
    # added by use.perl 2012-11-05 09:28:03
    PERL_VERSION=5.16.2