Using distcc with ports

[frankS2]

Hello there!

I have searched high and low after solutions for how i can use distcc with ports, very little information is found on this subject on google etc. So I am trying my luck in here, anyone have an how-to or some input on how I can do this here?

With regards
Frank

[DutchDaemon]

Search these boards; you will find a few threads mentioning distcc.

[frankS2]

Yes, mentioning it - but not how to set it up, thanks for you reply

[mousaka]

You've set up distcc correctly, right? Else read /usr/ports/devel/distcc/pkg-message.

Additionanlly I use sysutils/bsdadminscripts (http://www.freshports.org/sysutils/bsdadminscripts/) which provides the buildflags utility. This enables me to use distcc for port it works and disable it for others.
The file buildlfags.conf.sample was a good starting point for me.

mousaka

[frankS2]

Thanks for your reply mousaka - :-)

I have installed bsdadminscripts and set up the conf now. How can i make "portinstall" use the configfile? Does not seem like it uses it now

[mousaka]

Did you put these line in your /etc/make.conf (was /etc/rc.conf)?
# /usr/local/etc/buildflags.conf
BUILDFLAGS= /usr/local/share/bsdadminscripts/buildflags.mk
.if exists(${BUILDFLAGS})
.include "${BUILDFLAGS}"
.endif

portinstall should work fine, expect you have chosen one port that is excluded to use distcc in buildflags.conf ;)

mousaka

[frankS2]

mousaka, rc.conf i thought that was going to make.conf

hehe

[frankS2]

Here is my config files - still not compiling with distcc:
titanium# cat /usr/local/etc/buildflags.conf

# ---< configure ports >-------------------------------------------------------
/usr/ports/*{
# Clustering
SUBTHREADS= 6
USE_DISTCC
USE_CCACHE

# No distcc or threading.
*/archivers/p7zip {!SUBTHREADS}
*/audio/cdparanoia {!SUBTHREADS}
*/audio/libsndfile {!SUBTHREADS}
*/audio/nas {!SUBTHREADS}
*/converters/libiconv {!SUBTHREADS}
*/devel/autoconf261 {!SUBTHREADS}
*/devel/doxygen {!SUBTHREADS}
*/devel/gperf {!SUBTHREADS}
*/devel/libthai {!SUBTHREADS}
*/devel/nasm {!SUBTHREADS}
*/devel/ncurses {!SUBTHREADS}
*/devel/ORBit2 {!SUBTHREADS}
*/devel/pth {!SUBTHREADS}
*/dns/libidn {!SUBTHREADS}
*/editors/vim {!SUBTHREADS !USE_CCACHE}
*/games/ultimatestunts {!SUBTHREADS}
*/graphics/libafterimage{!SUBTHREADS}
*/graphics/libart_lgpl {!SUBTHREADS}
*/lang/ruby18 {!SUBTHREADS}
*/multimedia/ffmpeg {!USE_CCACHE !USE_DISTCC}
*/multimedia/mplayer {!SUBTHREADS}
*/multimedia/mencoder {!SUBTHREADS}
*/print/ghostscript* {!SUBTHREADS}
*/print/scribus {!USE_CCACHE}
*/print/xdvik {!USE_CCACHE}
*/security/libgpg-error {!SUBTHREADS}
*/security/nss {!SUBTHREADS}
*/science/hdf5 {!SUBTHREADS}
}

titanium# cat /etc/make.conf

# /usr/local/etc/buildflags.conf
BUILDFLAGS= /usr/local/share/bsdadminscripts/buildflags.mk
.if exists(${BUILDFLAGS})
.include "${BUILDFLAGS}"
.endif

# added by use.perl 2009-02-12 22:21:51
PERL_VER=5.8.9
PERL_VERSION=5.8.9


titanium# cat /usr/local/share/bsdadminscripts/buildflags.mk
#
# Copyright (c) 2006, 2007, 2008
# Dominic Fandrey <kamikaze@bsdforen.de>
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# version=1.9

# Default locations.
BUILDFLAGS_PARSER?= /usr/local/share/bsdadminscripts/buildflags.awk
BUILDFLAGS_CONF?= /usr/local/etc/buildflags.conf
BUILDFLAGS_TMP?= /tmp/buildflags.tmp.mk.${USER}
BUILDFLAGS_GCC_CC?= /usr/local/bin/gcc
BUILDFLAGS_GCC_CXX?= /usr/local/bin/c++
BUILDFLAGS_GCC_CPP?= /usr/local/bin/cpp

BUILDFLAGS_DISTCC?= /usr/local/bin/distcc
BUILDFLAGS_CCACHE?= /usr/local/bin/ccache

.if exists(${BUILDFLAGS_CONF})
# Parse configuration into a make file.
BUILDFLAGS!= test "${BUILDFLAGS_TMP}" -nt "${BUILDFLAGS_CONF}" || "${BUILDFLAGS_PARSER}" "${BUILDFLAGS_CONF}" > "${BUILDFLAGS_TMP}"

# Include that make file.
.if exists(${BUILDFLAGS_TMP})
.include "${BUILDFLAGS_TMP}"
.endif
.endif

# Use a different version of gcc.
.if defined(WITH_GCC)
BUILDFLAGS_GCC?= ${WITH_GCC}
.if exists(${BUILDFLAGS_GCC_CC}${BUILDFLAGS_GCC})
CC:= ${BUILDFLAGS_GCC_CC}${BUILDFLAGS_GCC}
CXX:= ${BUILDFLAGS_GCC_CXX}${BUILDFLAGS_GCC}
CPP:= ${BUILDFLAGS_GCC_CPP}${BUILDFLAGS_GCC}
.endif
.endif

# Use distcc.
.if defined(USE_DISTCC) && !${CC:M*distcc*} && exists(${BUILDFLAGS_DISTCC}) && !(defined(USE_CCACHE) && !${CC:M*ccache*} && exists(${BUILDFLAGS_CCACHE}))
CC:= ${BUILDFLAGS_DISTCC} ${CC}
CPP:= ${BUILDFLAGS_DISTCC} ${CPP}
CXX:= ${BUILDFLAGS_DISTCC} ${CXX}
.endif

# Use ccache.
.if defined(USE_CCACHE) && !${CC:M*ccache*} && exists(${BUILDFLAGS_CCACHE}) && !(defined(USE_DISTCC) && !${CC:M*distcc*} && exists(${BUILDFLAGS_DISTCC}))
CC:= ${BUILDFLAGS_CCACHE} ${CC}
CPP:= ${BUILDFLAGS_CCACHE} ${CPP}
CXX:= ${BUILDFLAGS_CCACHE} ${CXX}
.endif

# Use ccache and distcc.
.if defined(USE_CCACHE) && !${CC:M*ccache*} && exists(${BUILDFLAGS_CCACHE}) && defined(USE_DISTCC) && !${CC:M*distcc*} && exists(${BUILDFLAGS_DISTCC})
CC:= env CCACHE_PREFIX=${BUILDFLAGS_DISTCC} ${BUILDFLAGS_CCACHE} ${CC}
CPP:= env CCACHE_PREFIX=${BUILDFLAGS_DISTCC} ${BUILDFLAGS_CCACHE} ${CPP}
CXX:= env CCACHE_PREFIX=${BUILDFLAGS_DISTCC} ${BUILDFLAGS_CCACHE} ${CXX}
.endif

# Activate parallel builds for child makejobs.
.if defined(SUBTHREADS) && !(make(*install) || make(package))
MAKE_ARGS:= -j${SUBTHREADS}
.endif

# Activate normal parallel builds.
.if defined(THREADS)
.MAKEFLAGS: -j${THREADS}
.endif

[kamikaze]

mousaka, rc.conf i thought that was going to make.conf

heheIt does go into make.conf.

Some ports unfortunately ignore CC maybe that's the problem here.

After you try compiling, do you have a file /tmp/buildflags.tmp.mk.root? This basically should contain your buildflags.conf converted into make syntax. If it does not exist something is wrong with the include in your make.conf.

[frankS2]

Thanks for your reply kamikaze, i see mousaka pasted this to go in make.conf

# /usr/local/etc/buildflags.conf <---- Should I uncomment this one you think? or do it require some more?

BUILDFLAGS= /usr/local/share/bsdadminscripts/buildflags.mk
.if exists(${BUILDFLAGS})
.include "${BUILDFLAGS}"
.endif

And yes i had the /tmp/buildflags* files, so it seems fine :)

[kamikaze]

No, don't uncomment that.

That the files in /tmp exist proves that they're used. Why do you assume that the config file is not used?

[frankS2]

got it working now :) Have no idea what happend it just started working, heh..