Upgrade from source, using r/o NFS shares and cross-compilation

[zirias@]

A simple way to upgrade from source using builds done on a different machine is to just share /usr/src and /usr/obj with NFS and only execute the installkernel and installworld make targets on the target machine. This is really straight-forward, as long as both machines use the same (pre-upgrade) ABI, including the same target architecture.

Otherwise, you'll run into problems because some tools are built targeting the build machine; these are used during further building and during installation. So for example, if you build on amd64 targeting i386 (super easy with TARGET=i386), the result won't be installable with the "NFS trick" mentioned above on an actual i386 machine because e.g. make installworld will try to execute tools there that were built for amd64.

My current workaround is building for i386 twice. Once as a cross-build, using the result to populate a jail for poudriere as well as another "i386 builder" jail, which I use to build for the second time (to a different /usr/obj without cross-building). From this second build, installation on an actual i386 machine works fine.

Well, my question is: is there a (hopefully simple) way to just tell the build-system to also cross-compile these temporary tools, so I don't need this workaround building everything twice?

 

[mer]

Isn't /usr/obj the default output location for building? If so, then I would expect it to collide if you are building to the same /usr/obj on different architectures.
The NFS trick is nice when everything is the same architecture, you simply build on one machine then use the NFS mounts to install from.
If the target machine does not have the physical space for a local /usr/obj, I would look for a knob that allows you to specify the output directory:
/usr/obj/amd64 or /usr/obj/i386 as the top level instead of /usr/obj

 

[zirias@]

Isn't /usr/obj the default output location for building? If so, then I would expect it to collide if you are building to the same /usr/obj on different architectures.

Uhm, the actual world and kernel are built to architecture-specific subdirs, so this isn't a problem. BUT these temporary build tools (like e.g. make itself) exist only once. That's why I wrote I use a completely separate /usr/obj in my i386 jail doing the second ("native") build.

What I have works fine, but always building twice is a waste of both CPU time and disk space, so I wonder whether there's some way to cross-build the temporary tools as well

 

[mer]

Uhm, the actual world and kernel are built to architecture-specific subdirs,

Ahh, it's been a long time since I've built from source and couldn't remember.
As to the temporary tools, do they really only exist once? I would have thought they exist until the source gets updated causing a build to regenerate them or do a "make clean".

 

[zirias@]

As to the temporary tools, do they really only exist once?

Uhm, I was wrong on that, they are indeed inside the architecture-specific subdir. BUT, they always target the build system, not the target system. See for example:

# file /usr/obj/usr/src/i386.i386/tmp/usr/bin/objcopy
/usr/obj/usr/src/i386.i386/tmp/usr/bin/objcopy: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 13.0 (1300139), FreeBSD-style, not stripped

So, I might get away with only one /usr/obj tree, but I'd still have to build twice. Or is there a target that only builds these temp tools?

 

[covacat]

what if you make install from the building machine to a exported location and then on the target machine just rsync

 

[mer]

Zirias That is interesting. On one hand it makes sense; using a cross compiler on the host simply uses the tools to create output in the desired target, but some tools are indeed running under the host arch.
I don't know the answer to your question.

 

[zirias@]

what if you make install from the building machine to a exported location and then on the target machine just rsync

I tried installing onto an NFS share, this doesn't work because chflags(1) isn't (fully) supported. Sure, using rsync(1) might solve this, but I'd worry it might just update files it shouldn't (like everything below /etc which should be handled by etcupdate(8) instead).

 

[SirDice]

Or is there a target that only builds these temp tools?

make toolchain perhaps? I'm not sure what this target builds. I suspect you only have to build this for your target architecture.

     toolchain            Create the build toolchain needed to build the rest
                          of the system.  For cross-architecture builds, this
                          step creates a cross-toolchain.

 

[zirias@]

make toolchain perhaps? I'm not sure what this target builds. I suspect you only have to build this for your target architecture.

Definitely worth a test, will do so when I build an upgrade the next time! Although it might just build compiler/linker and not the other temp tools, but I'll see

 

[Alain De Vos]

You must differentiate between cross-compile and install.
A. The kernel.
B. The userland/world
C. The kernel-modules.
D. The ports

 

[zirias@]

Uhm, what? This is about the base system, and the source tree supports cross-compiling (which, of course, needs some tools targeting the build machine). "Base system" includes kernel, modules and userland.

Ports are out of scope here, they're build "natively" (no issue with i386 on an amd64 machine) in a jail. For "real" foreign archs, qemu can help, and there's an option to build and install "native xtools".