Solved Is it possible to buildworld without base compilers to use package compilers

Is it possible to buildworld without Clang or GCC, so a newer llvm or Clang can be installed as a package? I tried this, and now I'm stuck with a system that can't compile anything, from the package compilers.

This topic seems to belong in "Installing and Upgrading FreeBSD..." but its about ports too.

I may have to reinstall everything from scratch.
 
I don't think so, you'll need a compiler to... well, compile things ;)

From what I gathered the first thing the source tree does is to build the tools needed to compile, then the next phase is to use those tools to build the actual environment itself. But just so I understand correctly: you removed /usr/bin/clang? I assume you used the WITHOUT_CLANG directive?

You may not be able to compile anything, but I wager pkg is still working? You could always force a binary installation of, for example, devel/llvm39 and then use that to restore the base system.

(edit):

I almost forgot: be sure to edit /etc/make.conf accordingly!

So for example you might need this:
Code:
CC=clang39
CPP=clang-cpp39
CXX=clang++39
This should point your system to the Clang version from the ports, from there on you can restore the rest.
 
Yes, I tried devel/llvm39 from packages. That was my intent.

I'm trying GCC too. PCC doesnt seem recognized at all from make.conf.

Here's what I tried from make.conf
Code:
CC=/usr/local/bin/clang39
CXX=/usr/local/bin/clang++39
CPP=/usr/local/bin/clang-cpp39

Perhaps I need to bootstrap it, or I took out way too many options from src.conf.

In any case, its good to keep freebsd-update in, to be able to quickly back out of it. I left it out, so a custom install doesn't easily get overwritten, which isn't necessary to do.
 
You probably don't need to specify full pathnames, just the name will do. Assuming those files are in your search path of course. I've been using Clang from the time GCC was still the default and I always simply specified options like "CC=clang" which would be enough to get the system to use it (it's also the reason I could find it so quickly, I have the options commented out in /etc/make.conf these days, but left it there as a reminder on my LAN server ;)).

But bottom line: I don't think you'll need to reinstall from scratch.
 
I think it's, because I left out CPP from src.conf.

It looks like I need the full directory, when compiling the base, or at least in this case. It doesn't seem to matter for building ports.

When running make buildworld the error was:
Code:
clang-3.9: error: unable to execute command: Executable "ld" doesn't exist!
clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)
There is a different ld in /usr/local/bin/ld, so, I used a softlink from /bin/ld.

Now the error from buildworld, is similar to that of buildkernel:
Code:
make[ ] exec(objcopy) failed (No such file or directory)
*** Error code 1

When trying to compile different ports, I got various errors: one pertaining to C++, was that /lib/cpp "fails sanity check", or that some libraries aren't found. Some simple programs that only need C actually compile.
 
I went into /usr/src/usr.sbin/freebsd-update/, and ran make freebsd-update; then make install. The command freebsd-update appeared. It wanted "SERVERNAME" by command line or config file, so I copied /src/etc/freebsd-update.conf to /etc/.
 
That was helpful.
Code:
pkg which
was run a few times for objcopy and other files, to be softlinked from the missing files in /usr/bin/ to the files in /usr/local/bin/. devel/byacc, had to be installed for one. So far, there were too many problems and errors that kept showing up, once each one was taken care of.

A would have been plus side to this is, without compilers in the base system, make buildworld compiles in 2 hrs instead of 4, to would have later select a preferred llvm version.

freebsd-update didn't work here either, to force a re-download to a working system, that, or I messed the files up.

PCC isn't made for this, because it seems there is C++ code, or other dependencies, which it doesn't take care of.

At this point, this is too complicated for me, so I'll reinstall it. Thanks, ShelLuser.
 
Back to this situation. Some of what needed to be done is copy from an online repository or install cd from its /usr/bin/ directory the missing files from the displayed compiling error message to root /usr/bin/.
 
In src.conf(5), there is WITH_CLANG and WITH_CLANG_BOOTSTRAP (also with WITHOUT arguments). Both say they build the Clang C/C++ compiler, but one says it builds during the normal phase of the build, and the other says it builds during the bootstrap phase of the build. Does it build twice if both default settings are enabled, or is this an error in documentation that one is the bootstrap without the compiler, and the other is the compiler?

It looks like I can build a system that can compile by using WITH_CLANG_BOOTSTRAP, and WITHOUT_CLANG, along with a newer Clang from packages. Also, without needing: XCC/XCXX/XCPP. This also requires in src.conf (some of which are obvious): TOOLCHAIN, BINUTILS, BINUTILS_BOOTSTRAP, INCLUDES, LLD, CXX and CPP.
 
make.conf for building kernel, base and ports:
Code:
CC=    /usr/local/bin/clang40
XCC=   /usr/local/bin/clang40
CXX=   /usr/local/bin/clang++40
XCXX=  /usr/local/bin/clang++40
CPP=   /usr/local/bin/clang-cpp40
XCPP=  /usr/local/bin/clang-cpp40
As for binutils, I rather go with from base, than devel/binutils. Here are llvm40 equivalents of binutils:
Code:
#XAS=       /usr/local/bin/llvm-as40
#XAR=       /usr/local/bin/llvm-ar40
#XLD=       /usr/local/bin/lld-link40
XNM=        /usr/local/bin/llvm-nm40
XOBJDUMP=/usr/local/bin/llvm-objdump40
#XRANLIB=   /usr/local/bin/llvm-ranlib40
XSTRINGS=   /usr/local/bin/llvm-strings40
#XOBJCOPY=
A few LLVM equivalents in place of binutil's executables caused builds to fail depending on what was being built. lld (llvm ld) is the replacement for ld, and it is not complete: https://wiki.freebsd.org/GPLinBase. lld-link40 and llvm-as40 fail to compile the kernel at this time. llvm-ar40, and llvm-ranlib40 fail with possibly minor errors for buildworld.

https://wiki.freebsd.org/ExternalToolchain doesn't mention XOBJCOPY, but it can be set. objcopy's equivalent is in base as ELFCOPY, mentioned in src.conf(5) of FreeBSD 11. ELFCOPY and ELFTOOLCHAIN are from https://sourceforge.net/projects/elftoolchain/. Commented out XAS, XAR, XLD and XOBJCOPY revert to base toolchain files in /usr/bin/.

devel/elfutils from https://sourceware.org/elfutils/ is a different and GNU project to replace binutils. It requires depreciated lang/gcc48, and devel/binutils. Filenames of elfutils are prefixed with eu-.
 
Last edited:
Based on assistance from other FreeBSD users: the X prefix isn't needed for compiling using external toolchains, and a different directory is needed for toolchain utils. X compiler and X util settings are for cross compiling, when a toolchain is in the base system, and another toolchain is used for a secondary purpose.
Code:
CC=        /usr/local/bin/clang40
CXX=       /usr/local/bin/clang++40
CPP=       /usr/local/bin/clang-cpp40

LD=        /usr/local/llvm40/bin/ld.lld
NM=        /usr/local/llvm40/bin/llvm-nm
OBJDUMP=/usr/local/llvm40/bin/llvm-objdump
STRINGS=   /usr/local/llvm40/bin/llvm-strings
These work for compiling kernel and base. At the moment, AS, AR, and RANLIB aren't completely functional.
I've used clang equivalents from /usr/local/llvm40/bin/ for CC, CXX, CPP as well.


Also, in some cases of rescue:
From /usr/src/, make toolchains, make kernel-toolchain, and make kernel-toolchains can help you back out of some errors to build a kernel or world. These make settings can be found in /usr/src/Makefile.

Another rescue caveat is to copy a working kernel directory, then edit /boot/loader.conf to include that kernel:
Code:
kernels="kernel kernel.old kernel.bk"
kernel= "kernel" # chosen kernel
 
Last edited:
Well, I got conclusive results. It's possible to do this but I wouldn't bother because it has the potential to 'break' your system, at least some parts of it.

So in the mean time I finally got around to customizing my new FreeBSD server and because the system already had devel/llvm40 installed I figured I could do without the base Clang version.

So I added this in /etc/src.conf:

Code:
WITHOUT_CLANG=
WITHOUT_CLANG_BOOTSTRAP=
... and then continued to build my entire system. Everything worked without any flaws. Building, installation, it's all perfect. Except it's not.

First thing which puzzled me was that Clang was still part of my base system while at the same time it also wasn't:

Code:
omicron:/home/peter $ file `which cc`
/usr/bin/cc: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 11.1, FreeBSD-style, stripped
omicron:/home/peter $ ls /usr/lib/clang
ls: /usr/lib/clang: No such file or directory
I didn't give this much thought until I started getting errors during the build of some ports. I was puzzled at first but soon discovered the origin of these errors.

This is while building security/nss:

Code:
In file included from ctr.c:17:
./rijndael.h:21:10: fatal error: 'emmintrin.h' file not found
#include <emmintrin.h> /* __m128i */
         ^~~~~~~~~~~~~
1 error generated.
gmake[5]: *** [../../coreconf/rules.mk:393: FreeBSD11.1_OPT.OBJ/FreeBSD_SINGLE_SHLIB/ctr.o] Error 1
gmake[5]: Leaving directory '/usr/ports/security/nss/work/nss-3.34/nss/lib/freebl'
Research learned me that the file emmintrin.h used to be located in /usr/lib/clang/4.0.0, a directory which is now removed.

The file is still located on my system, but its current location is /usr/local/llvm40/lib/clang/4.0.1/include/immintrin.h, a place where it apparently can't be found. Not too surprising if you ask me, especially when you keep this aspect in mind:

Code:
omicron:/home/peter $ cc -v
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
Target: x86_64-unknown-freebsd11.1
Thread model: posix
InstalledDir: /usr/bin
So then I concluded that this is giving me more trouble than it's worth, as such I decided I'd rebuild my base system, included Clang again, and all would be back to normal.

Yah, but there is a flaw in that line of reasoning, because although the base system will indeed bootstrap itself to build its own set of tools, it'll need a working compiler to compile its own stuff first.

And apparently I don't have one right now:

Code:
--- kerberos5/lib/libheimbase__L ---
ctfconvert -L VERSION -g heimbase.pico
--- null.pico ---
/usr/local/bin/ccache cc -target x86_64-unknown-freebsd11.1 --sysroot=/usr/obj/u
sr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin -fpic -DPIC -g -O2 -pipe -I/usr/src/cr
ypto/heimdal/include   -DHAVE_CONFIG_H -I/usr/src/kerberos5/include -MD  -MF.dep
end.null.pico -MTnull.pico -std=gnu99 -fstack-protector-strong    -Qunused-argum
ents  -c /usr/src/crypto/heimdal/base/null.c -o null.pico
--- secure/lib/libcrypto__L ---
ctfconvert -L VERSION -g ccm128.o
--- lib/liblzma__L ---
In file included from /usr/src/contrib/xz/src/liblzma/lz/lz_encoder.c:23:
/usr/src/contrib/xz/src/liblzma/common/memcmplen.h:19:11: fatal error: 'immintri
n.h' file not found
#       include <immintrin.h>
                ^~~~~~~~~~~~~
1 error generated.
*** [lz_encoder.o] Error code 1

make[4]: stopped in /usr/src/lib/liblzma
1 error

make[4]: stopped in /usr/src/lib/liblzma
*** [lib/liblzma__L] Error code 2

make[3]: stopped in /usr/src
--- secure/lib/libcrypto__L ---
ERROR: ctfconvert: rc = -1 No entry found [dwarf_next_cu_header_c(61)]
A failure has been detected in another branch of the parallel make
As you can see it's complaining about the exact same file as before.

Therefor this leads me to conclude that it is indeed perfectly possible to build your base system without Clang and without having it bootstrap Clang (the latter is an interesting option because it can save you quite some build time).

However, it can result in a few issues as showcased here which lead up to me concluding that in the end this isn't worth the hassle for regular use.

I think it will be more useful to try and install a 'dummy' devel/llvm40 port and then try to build those ports which claim to rely on this. So basically tricking them to build against the compiler in the base system and then see what happens next.

The main goal is to save time of course (not having to build Clang twice), but it could also be quite education.

Anyway, hope this can give some people some useful information.
 
It worked for me without base clang and clang bootstrap. All of my ports built (except Rust and Firefox), so did kernel and base. For binutils replacements, all but the excluded options in make.conf from my latest post above worked for everything.

Code:
CC=        /usr/local/bin/clang40
CXX=       /usr/local/bin/clang++40
CPP=       /usr/local/bin/clang-cpp40

LD=        /usr/local/llvm40/bin/ld.lld
NM=        /usr/local/llvm40/bin/llvm-nm
OBJDUMP=/usr/local/llvm40/bin/llvm-objdump
STRINGS=   /usr/local/llvm40/bin/llvm-strings
These work for compiling kernel and base. At the moment, AS, AR, and RANLIB aren't completely functional.
I've used clang equivalents from /usr/local/llvm40/bin/ for CC, CXX, CPP as well.
Code:
CC= /usr/local/llvm40/bin/clang
CXX=/usr/local/llvm40/bin/clang++
CPP=/usr/local/llvm40/bin/clang-cpp
Firefox and Rust were the only programs that had problems. Perhaps these programs including NSS have Mozilla in common. However, I didn't have problems building NSS. Firefox and Rust have had problems anyway building from package/ports clang without using the methods in this thread.

So it would be a showstopper for persons needing to rebuild, say, Xorg or nvidia after the installworld?
Xorg (and Radeon) worked for me with this method. IDK about Nvidia.

Edit: everything from gtk2, gtk3, qt4, Apache openoffice, vlc-qt4 built with this for me. Compact c file format in src.conf is also required for all of this to work.
 
Last edited:
So it would be a showstopper for persons needing to rebuild, say, Xorg or nvidia after the installworld?
I can't be conclusive about every port because there were also a few which build normally, but several others amongst which security/nss as well as the base system itself constantly gives me errors.

One way to solve this is to define cc and others in /etc/make.conf and then explicitly point it to clang or clang-4.0 (so: /usr/local/llvm40/bin/clang-4.0). Another option can be to create cc as a link in /usr/local/bin and then point that to the right direction while also ensuring that the search path will have /usr/local/bin before /usr/bin (which is not a desirable situation in my opinion).

This setup is definitely broken because /usr/bin/cc gets left behind and at the time of writing results in a version conflict. In other words: the still left behind base compiler version is 4.0.0 while the only available include files are from the Port which is version 4.0.1. That obviously doesn't match.
 
Code:
COMPILER_TYPE=clang
CC= /usr/local/bin/clang50
CXX=/usr/local/bin/clang++50
CPP=/usr/local/bin/clang-cpp50
LD= /usr/local/llvm50/bin/ld.lld
#LD=/usr/local/bin/ld # for binutil's ports ld
#default is base utils
devel/xtoolchain-llvm-devel and another xtoolchain port of your architecture (devel/xtoolchain-llvm50) and LLVM will install examples for cross compiling in /local/share/toolchains. The above is based on that, without the X or CROSS_compile prefix.

Update: About 100 ports, down from thousands don't compile with LLVM's lld on amd64 https://wiki.freebsd.org/action/diff/GPLinBase?action=diff&rev1=226&rev2=227.

For an LD that should be able to build everything, either use the one in base which is default, or install the one from devel/binutils to reference it.

I'm trying to install all utils and clang from ports, so this will take more steps, and may not be able to install everything. The ports for this are devel/llvm50 and devel/binutils.

For fully using ports, without a base utils and without base clang, these arguments and their ports LLVM or binutils equivalents are needed in /etc/make.conf:
Code:
NM=      /usr/local/bin/nm
OBJDUMP= /usr/local/bin/objdump
AS=      /usr/local/bin/as
AR=      /usr/local/bin/ar
RANLIB=  /usr/local/bin/ranlib
OBJCOPY= /usr/local/bin/objcopy
STRINGS= /usr/local/bin/strings
STRIP=   /usr/local/bin/strip
SIZE=    /usr/local/llvm50/bin/llvm-size
I will try different combinations of LLVM from /usr/local/llvm50/, and binutils from /usr/local/bin/ for when a missing file is asked for with the arguments for that file. Also, I will use LLVM's utils when https://wiki.freebsd.org/GPLinBase says it is completed, and devel/binutils files for the remainder.
 
I know this thread is getting old but I got a new chapter to add, and I'm 80% sure that the problems I encountered today can be traced right back to the above test. Bottom line though: this is definitely not worth the effort, even now.

Yesterday I couldn't play sound on my server and (mistakenly!) assumed that this was caused by me not having the sound module(s) installed (I use MODULES_OVERRIDE in /etc/make.conf). So the idea was simple: add sound and rebuild the kernel. Yeah... about that: every time I tried to start cc I was met with an error message (signal 4, "Illegal instruction.") and it also dumped core on me. "Not good".

I'm not sure what happened because I've performed another system update between the test I mentioned above (plus its repair afterwards) and now and I've never had any issues with building ports. Even so, I can only conclude that this is yet another effect from removing Clang, "fixing" it by copying parts of the port into the base system and then trying to rebuild again.

Anyway, today I fixed the problem and I'm pretty sure I'm now in the clear. I did what I should have done in the first place: rely on my Jail. So I mounted my source tree (nullfs) into my (pretty pristine) jail, I copied my configuration and everything and then build the world and the kernel. Copied /usr/obj/usr back onto the host and then re-installed both the kernel and the world.

Well, right now I'm typing this on my server in my trusty KDE environment, all is right with the base once more. Yet I am convinced that the cause of my problem can be directly traced back to this stunt, as well as me performing a somewhat failed attempt in "repairing" it (instead of relying on a Jail I basically hacked the base system which, when looking back, wasn't the best of ideas. Even though it worked at first).

I'm tempted to compare the removal of Clang with the removal of crypto (WITHOUT_CRYPT and/or WITHOUT_OPENSSL in /etc/src.conf). That too breaks more than it's worth, even though you might have all the required libraries and modules installed from the Ports collection.

And the main problem is that you may only notice that something is wrong at a later time.
 
According to src.conf(5) in FreeBSD 11.2, ELFTOOLCHAIN_BOOTSTRAP has addr2line, nm, size, stings, and strip. ELFCOPY as OBJCOPY should be the default. It seems there are different tools in many bootstraps compared to the corresponding set, that are together often used for builds from the kernel, world and ports.

I built it without CLANG and GCC base compiler in the past, and package CLANG built almost everything I used except Firefox and Thunderbird. What didn't work, is when I tried to replace bootstraps, binutils, includes, and other toolchain utils. LD or LLD is in the cross-compiler with the package compiler, and that worked for most ports.

These settings are provided with the package compiler.
Code:
COMPILER_TYPE=  clang
X_COMPILER_TYPE=clang
CC=    /usr/local/bin/clang60
XCC=   /usr/local/bin/clang60
CXX=   /usr/local/bin/clang++60
XCXX=  /usr/local/bin/clang++60
CPP=   /usr/local/bin/clang-cpp60
XCPP=  /usr/local/bin/clang-cpp60
LD=    /usr/local/llvm60/bin/ld.lld
XLD=   /usr/local/llvm60/bin/ld.lld
CROSS_BINUTILS_PREFIX=/var/empty
The X prefix equivalent can be found by installing devel/xtoolchain-llvm60, and then looking in /usr/local/share/toolchains/. You may need both.

Leave the bootstraps in, except GCC and these, plus objcopy, will be from base's elftoolchain bootstrap. Bootstraps are required for port and other builds.
Code:
# leave these and their X prefix equivalent out, elftoolchain and its bootstrap in base cover this
#NM=
#OBJCOPY=
#STRINGS=
#STRIP=
#SIZE=

This is what's left, and I don't know if the pkg/ports binutils can cover for this:
Code:
# also their X prefix equivalent
#OBJDUMP=
#AS=
#AR=
#RANLIB=
Leave these out of make.conf, and include binutils and its bootstrap for src.conf.

I believe the arguments used that are not defaults from base, need to be included and repeated with an X in front of the arguments, ie: XCC, XCXX, XOBJDUMP
 
I got the kernel and base to install without the base compilers.
make.conf:
Code:
COMPILER_TYPE=  clang
CC=    /usr/local/bin/clang60
CXX=   /usr/local/bin/clang++60
CPP=   /usr/local/bin/clang-cpp60
LD=    /usr/local/llvm60/bin/ld.lld
This sets compiling of ports, kernel and base. The X prefix only overrides these for kernel and base, to allow ports to use a different compiler, typical in cross compiling, so it can be left out. The CROSS_BINUTILS_PREFIX= setting is only needed for compiling the kernel or base in a different location than typical builds for cross compiling.

Because I have these set in make.conf to a C compiler and linker in packages, I have removed the equivalent settings in src.conf:
Code:
WITHOUT_CLANG=yes
WITHOUT_CXX=yes
WITHOUT_CPP=yes
WITHOUT_LLD=yes
# redundantly removed GCC related components
WITHOUT_GCC=yes
WITHOUT_GCC_BOOTSTRAP=yes
All bootstraps left in for: BINUTILS, CLANG, ELFTOOLCHAIN, LLD. Many are required.

This worked for installing the kernel and world. I don't expect this to work for Thunderbird and Firefox, but I expect it to work for most ports. For Thunderbird, Firefox and other ports, I suggest a trial of leaving LD in base, and leaving that as is from make.conf.

To check for yourself on which compiler and toolsets are being used, run make buildworld with grep, then cancel it with ctrl-c.

Unsure if package devel/xtoolchain-llvm60 is needed.
 
For Thunderbird, Firefox and other ports, I suggest a trial of leaving LD in base, and leaving that as is from make.conf
By "leaving that as is from make.conf," do you mean deleting the line "LD= /usr/local/llvm60/bin/ld.lld" from the make.conf above? I am yet to build Firefox, Palemoon and so on because of the wrong entries in the make.conf (asme as yours though). Kindly shed more light on your recommendation.
 
Back
Top