build a machine without development tools

PMc

Active Member

Reaction score: 43
Messages: 133

I am wondering if there is a clean way to build a system without the compiler and linker stuff.

I see no point in having these around if the respective machine will only run an application (that is built elsewhere) and never be used to build anything. And most commercial unix systems were delivered in that way (if you would not explicitely order a development system.)

Nevertheless, I do build my systems from local sources, following a procedure similar to the one described in the handbook, i.e. build it all on a development machine (within a chroot, with all the proper configs applied), then dump /usr/src and /usr/obj to the destination and there do "make installworld" etc.

I tried to kick out the toolchain stuff via /etc/src.conf - this worked, I got the /usr/obj tree created and could install from it, but now I can not even remake my sendmail config in /etc/mail:

Code:
# make
sh: cc: not found
make: "/usr/share/mk/bsd.compiler.mk" line 147: Unable to determine compiler type for CC=cc.  Consider setting COMPILER_TYPE.
It seems the include files for make want to see a compiler, even where none is needed. I am certain I will run into greater trouble when I try to do another make installworld...
Certainly this can be fixed by twiddling with proper settings for make - but if this tends to become laboursome, then it's not worth the effort (and new surprizes may appear after every version update).

Anyway, I think such an option should exist - why would people want to build and install the development stuff onto their dedicated production arrays?
 

olli@

Active Member
Developer

Reaction score: 221
Messages: 246

I am wondering if there is a clean way to build a system without the compiler and linker stuff.

I see no point in having these around if the respective machine will only run an application (that is built elsewhere) and never be used to build anything.
Sure, you can do that via src.conf(5), as you mentioned. I do that myself for building jails.
but now I can not even remake my sendmail config in /etc/mail
If you want to exclude everything needed for building, then you should also exclude make(1), of course, via WITHOUT_MAKE in src.conf(5). I have several jails that don't have compiler, bin toolchain, include files, static libraries and a bunch of other things. They also don't have make(1). The files in /etc/mail are built on the host and then distributed across the jails where needed. Basically, /etc is read-only within those jails.

If you still want to rebuild the sendmail files, that can be done by removing the .include line at the end of the Makefile. I think its only purpose is to import the SHAREMODE variable. You can simply hardcode it by adding a line SHAREMODE=444.
 

Vull

Active Member

Reaction score: 58
Messages: 148

What I'm doing recently is building packages on the development system using pkg create -a, then I install only the selected top-level packages I want, onto a fresh FreeBSD install, which is to be used at the prototype for the production system I want, using commands like, for example, pkg add php73-7.3.2.txz. This hopefully and seemingly gets all of the runtime dependency packages, and none of the build dependency packages, onto the production prototype, where I can then prepare a production level set of packages using pkg create -a again, only this time from the production prototype instead of from the development system. The production system package archive is a fraction of the size of the development package archive. This is being partially automated with shell scripts and text file lists. The production level systems install very quickly from packages, and don't need to have the source tree, ports tree, or any of the build software. They require less than half the disk space and the packages install in very tiny fractions of the development build times, like for instance, 15 minutes to install from packages using a script and a list, for the same list of software that took 5-6 days to build from ports on the same slow machine.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 6,944
Messages: 28,894

You might want to have a look at ports-mgmt/poudriere or ports-mgmt/synth to automate package creation. Both can create a full local repository that can be used with pkg(8) (just set up a custom repository in /usr/local/etc/pkg/repos/).

Using a combination of Puppet and my own repositories I can (re)install a server in about 10 minutes.
 
OP
OP
PMc

PMc

Active Member

Reaction score: 43
Messages: 133

Wow, thanks for Your answers! :)

Building of ports is currently not my issue. I know about poudriere, but never looked into it, because the day poudriere arrived, I had already some (more or less) working scripts for port-building.

So, currently I am looking into the deployment of the base OS (mainly because people here told me "you should update more often"), and checking the steps that are uneasy and therefore prone to avoidance strategies. ;)

olli@ : Your approach is radical. The mailer-config issue is certainly neglectable, but, when removing make, one cannot do make installworld and make delete-old, so one has to do that on the development machine, and it becomes more or less an image deploy.
That makes certainly sense for a serious production farm - but if I get You right, You are doing this only for jails? (An image deploy for a standalone machine would be slightly more complicated.)

My plan is to have a central building instance that updates the source-tree and creates the /usr/src and /usr/obj for deployment to the targets (no matter if the target is a jail or a standalone machine). And in a sidestep I was just wondering if I really need to spend the extra 30 minutes to build the llvm stuff for each and every destination, or if I can get rid of that for the targets where it is just not needed.
 

Vull

Active Member

Reaction score: 58
Messages: 148

PMc - Thanks for your graceful reply. I made my earlier post in too much haste, without realizing that it doesn't address your opening question regarding the artifacts from building the base system. Clearly enough that's outside my experience, so please accept my apology for the earlier off-topic response.

(Edited to add: ) A little more on-topic perhaps, one might still use a similar strategy and prepare self-compiled packages to reload components such as llvm, when needed, and later delete it, thereby at least sidestepping the extra 30 minute compile for llvm.
 
OP
OP
PMc

PMc

Active Member

Reaction score: 43
Messages: 133

Hi @all,
I think I have found an apparently working solution, by adding the following to /etc/make.conf:

Code:
HOSTNAME_S != /bin/hostname -s
.if ${HOSTNAME_S} != mydevel
COMPILER_TYPE=none
LINKER_VERSION=0
LINKER_TYPE=none
CC=true
Ld=true
.endif
(The "hostname" stuff checks if we are on the building system or on the target system - obviousely these options are only to be set when installing, not when building the /usr/obj tree.)

Furthermore, the option WITHOUT_TOOLCHAIN= is not to use, because it provides the strip command which is needed during install. So what I currently have excluded from the development tools in /etc/src.conf (not fully understanding what is exactly what here):

Code:
WITHOUT_BINUTILS=
WITHOUT_CXX=
WITHOUT_CPP=
WITHOUT_CLANG=
WITHOUT_GCC=
WITHOUT_GDB=
WITHOUT_GNUCXX=
WITHOUT_LLD=
WITHOUT_LLDB=
 

sidetone

Daemon

Reaction score: 366
Messages: 1,014

I already did that. Thread is-it-possible-to-buildworld-without-base-compilers-to-use-package-compilers.59874.

For the linker, you can choose the one that comes with LLVM in packages, and that will build the base, kernel and most, but not all ports. Also, leaving the binutils linker in the base system is a good choice. I didn't link to the binutil's linker from packages from make.conf. Leave base binutils in, unless you want to use LLVM's linker and not be able to compile a few ports, or if you want to troubleshoot for each function not included in base's binutils. My system compiled successfully without the base compiler.
 

olli@

Active Member
Developer

Reaction score: 221
Messages: 246

olli@ : Your approach is radical. The mailer-config issue is certainly neglectable, but, when removing make, one cannot do make installworld and make delete-old, so one has to do that on the development machine, and it becomes more or less an image deploy.
That makes certainly sense for a serious production farm - but if I get You right, You are doing this only for jails? (An image deploy for a standalone machine would be slightly more complicated.)
I wouldn't call it “radical”. I think it's the standard procedure for doing things like that. Of course I do make installworld etc. on the host system (not within the jails) with appropriate DESTDIR=.... It doesn't matter much if those are jails, virtual machines or physical servers. For real servers, the installworld could be done via NFS. Or you install into an image and transfer that to the server. I'm sure there are even more ways to do it.
 
Top