Solved Installing cross-compiled world

tembun

tembun

I want to cross-compile buildworld and buildkernel on my local home server (LAN) and install the results on my laptop.
As far as I understood, the best and most recommended way to do this would be to set server MAKEOBJDIRPREFIX to say /usr/objcross and on my laptop mount it as /usr/obj (NFS). Is that correct? Would it be enough for a complete installation to do
Code: 
# make installkernel
# make installworld
on my laptop when the compilation is done on the server?

By the way, is NFS mount is the only possible way to install cross-compiled world? I looked in release(7) and build(7) and there are only ways to install the release using DVD, memstick or stuff like which sounds like a fresh installation. I'm looking for something that would allow me to quickly update my laptop system frequently. For example, I expected that release(7) would have some target that would include say a tar(1) archive where the results of buildworld and buildkernel were packaged and then something like install.sh (or a make(1) target), which would install them. In that case the Internet connection between building server and the target machine would not be necessary and I could just transfer .tar file over to my laptop and still quickly update my system.
 
tembun said:
I want to cross-compile buildworld and buildkernel on my local home server (LAN) and install the results on my laptop.
Click to expand...
Same architecture? You're not cross-compiling if the architecture is the same.

tembun said:
As far as I understood, the best and most recommended way to do this would be to set server MAKEOBJDIRPREFIX to say /usr/objcross and on my laptop mount it as /usr/obj (NFS).
Click to expand...
Don't need to set MAKEOBJDIRPREFIX, just NFS share /usr/obj and /usr/src (both can be read-only) and mount them on the other machine.

One thing to watch out for, make damn sure the date/time is correct and in sync on both machines. You can get some really weird errors if the time is too far off between the server and client.
 
SirDice said:
Same architecture? You're not cross-compiling if the architecture is the same.
Click to expand...
Yes, arch is the same. Sorry, my bad, I meant I am compiling on other machine.

SirDice said:
Don't need to set MAKEOBJDIRPREFIX, just NFS share /usr/obj and /usr/src (both can be read-only) and mount them on the other machine.
Click to expand...
Oh, yeah, it was so simple, why didn't I think about that? :)

I guess that I still don't want to put results in /usr/obj, because the server itself also compiles its system from source, so I want object files for the server itself and for my laptop to be in different places.

Thank you so much!
 
tembun said:
I looked in release(7) and build(7) and there are only ways to install the release using DVD, memstick or stuff like which sounds like a fresh installation.
Click to expand...
Documentation hasn't been updated yet, but there's pkgbase nowadays. That might be a good solution for you. You can build those pkgbase packages on one machine and install/update them on another.

tembun said:
because the server itself also compiles its system from source, so I want object files for the server itself and for my laptop to be in different places.
Click to expand...
Unless you used a different src.conf(5) and/or make.conf(5) for both systems those compiled targets will be exactly the same, so why do you think you need to separate them?
 
tembun said:
Server is running 16.0-CURRENT and my laptop - 15.0-STABLE. Would not object files overwrite each other in this case?
Click to expand...
In that case, yes. I assumed you wanted to use the same version on both systems.
 
tembun said:
Server is running 16.0-CURRENT and my laptop - 15.0-STABLE. Would not object files overwrite each other in this case? And wouldn't it confuse META_MODE or devel/ccache and break incremental builds?
Click to expand...
I do build different branches from source regularly. For this, every branch has its own source directory, resulting in their own OBJDIR automatically, e.g.:
Code: 
/usr/src-14.3
/usr/src-15.0
/usr/src-15stable
/usr/src-main

/usr/obj/usr/src-14.3
/usr/obj/usr/src-15.0
/usr/obj/usr/src-15stable
/usr/obj/usr/src-main
In my use case /usr/src-main is a deep clone on a ZFS dataset, from which a snapshot is taken and cloned to wanted dataset branches, then git switch <branch>.

/etc/src.conf for different build scenarios are managed by build(7) variable SRCCONF, pointing to custom configuration files: SRCCONF=/root/build/src-15.0.conf
 
T-Daemon said:
I do build different branches from source regularly. For this, every branch has its own source directory, resulting in their own OBJDIR automatically, e.g.:
Code: 
/usr/src-14.3
/usr/src-15.0
/usr/src-15stable
/usr/src-main

/usr/obj/usr/src-14.3
/usr/obj/usr/src-15.0
/usr/obj/usr/src-15stable
/usr/obj/usr/src-main
In my use case /usr/src-main is a deep clone on a ZFS dataset, from which a snapshot is taken, and cloned to wanted dataset branches, then git switch <branch>.

/etc/src.conf for different build scenarios are managed by build(7) variable SRCCONF, pointing to custom configuration files: SRCCONF=/root/build/src-15.0.conf
Click to expand...
Thank you for sharing this!

I've just done the very similar thing for my server: I have a separate directory for the sources that have to be built for my laptop. And I also have separate variants for /etc/src.conf, /etc/src-env.conf and /etc/make.conf for the laptop-build. I only didn't know that just the name of the directory with sources itself changes the nested path inside directory for object files (e.g. /usr/obj). So I added custom MAKEOBJDIRPREFIX into laptop-version of /etc/src-env.conf, but now I know that this is not the only option and I can theoretically share (sources for laptop are located in /usr/src-rio in my case) /usr/obj/usr/src-rio instead of /usr/obj-rio and mount it as /usr/obj/usr/src on my laptop.

I also wrote a small script for launching the build:
sh: 
#!/bin/sh

SRC=/usr/src-rio
MAKE_CONF=/etc/make-rio.conf
SRC_CONF=/etc/src-rio.conf
SRC_ENV_CONF=/etc/src-env-rio.conf

progname=$(basename "${0}" .sh)

err()
{
    echo "${progname}: ${@}" 1>&2
    exit 1
}

check_file()
{
    local file="${1}"
    [ -f "${file}" ] || err "${file} not found"
}

[ -d "${SRC}" ] || err "${SRC} not found"
for conf_file in ${MAKE_CONF} ${SRC_CONF} ${SRC_ENV_CONF}; do
    check_file "${conf_file}"
done

kldload -n filemon || err "Can't load filemon(4) kernel module"
make -C "${SRC}" \
    __MAKE_CONF="${MAKE_CONF}" \
    SRCCONF="${SRC_CONF}" \
    SRC_ENV_CONF="${SRC_ENV_CONF}" \
    -j $(sysctl -n hw.ncpu) \
    buildworld \
    buildkernel
 
Well, I tried the abovementioned solution and I have a little issue: on my server directories for laptop source and object files are called /usr/src-rio and /usr/obj-rio, but on my laptop I mount them as /usr/src and /usr/obj. When I try to make installkernel KERNCONF=LEAN on the laptop, I got an error:
Code: 
/usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile:5248: Cannot open /usr/src-rio/sys/conf/kern.post.mk
and in /usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile I have:
Makefile: 
S=/usr/src-rio/sys
and this is the error: prefix /usr/src-rio was generated because on my server sources are located there, but on my laptop it should be /usr/src instead. This is in fact the second iteration of the problem, because I already figured that I have to set in /etc/src-env-rio.conf:
Makefile: 
MAKEOBJDIRPREFIX=/usr/obj-rio
OBJROOT=/usr/obj-rio/usr/src/
because without that I also had conflicts with directory names (because they have different locations on these two machines).

I had a look at /usr/src/share/mk/src.sys.obj.mk but it's still not very clear to me how should I fix this issue for names to be generated properly. Could anyone please help me with that?
 
You are complicating the issue. The easiest way to keep it simple, is to create on the server and laptop the same source directory names and mount the NFS shares accordingly. Remove "MAKEOBJDIRPREFIX" , "OBJROOT" and all the other variables regarding source and obj locations.

Server:
Code: 
/usr/obj
/usr/src-rio

NFS /etc/exports example:
Code: 
/usr/obj            -ro -network 192.168.1.0/24
/usr/src-rio        -ro -network 192.168.1.0/24



Laptop directories:
Code: 
/usr/obj
/usr/src-rio

NFSv3 mounting shares (NFSv4: set "V4:", mount accordingly the NFS V4 root and set option "nfsv4" ) :
Code: 
# mount 192.168.1.10:/usr/obj  /usr/obj
# mount 192.168.1.10:/usr/src-rio  /usr/src-rio

Make sure /etc/src.conf (or custom src.conf) is the same on both machines, eventuall set SRCCONF.
Code: 
/usr/src-rio # make installworld SRCCONF=/root/build/src-rio.conf

What is the laptops file system? UFS of ZFS? If it is ZFS, you may want to create a ZFS boot environment and DESTDIR installkernel installworld to the mounted BE.
 
Thank you T-Daemon!

I suspected that using the identical names on both machines would be a simpler way for this (and I tried it and it does work), but for me it's very interesting to try different solutions (just want to gain more experience), so I wonder can I accomplish that (different names for src and obj) by adjusting some make(1) variables or the very build(7) system is not designed for such tweaks?
 
tembun said:
When I try to make installkernel KERNCONF=LEAN on the laptop, I got an error:
/usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile:5248: Cannot open /usr/src-rio/sys/conf/kern.post.mkand in /usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile I have:
S=/usr/src-rio/sysand this is the error: prefix /usr/src-rio was generated because on my server sources are located there, but on my laptop it should be /usr/src instead.
Click to expand...
I looked a bit deeply:
as I understand, /usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile is generated by this line in Makefile.inc1, i.e. by config(8). This program by default (if no -s option specified) uses current directory as the root of the kernel source tree. And then it writes that string S=<srcdir> into generated Makefile. So, in Makefile.inc1 config(8) is executed without -s, so in order to have a correct path for S in /usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile I have to set -s /usr/src/sys. But there's no a separate variable in Makefile.inc1 for this. So, I guess that my only solution is to patch the Makefile.inc1 and introduce a make(1) variable for -s option there?
But will installworld have the similar problem with generated paths too?
 
tembun said:
So, in Makefile.inc1 config(8) is executed without -s, so in order to have a correct path for S in /usr/obj/usr/src/amd64.amd64/sys/LEAN/Makefile I have to set -s /usr/src/sys.
Click to expand...
Oh, no, I was wrong: if I set -s /usr/src/sys for config(8) then it indeed will write S=/usr/src/sys in Makefile, but the problem is it will also use /usr/src/sys as source directory for compiling files too, and this is clearly wrong.

I really start to suspect that this scenario with compiling objects that are supposed to be used under different directory names on different machine is not really well supported by default. Patching config(8) sources for introducing variable for alias for S variable in resulting Makefile is my another guess.

Have not anybody tried to do something similar?
 
I just thought: maybe I should instead set up a jail for building for laptop, where source and object directories would be named /usr/src and /usr/obj? I've never worked with jails before, will it be a good place to start? Also, will I be able to export NFS /usr/src and /usr/obj from jail?
 
tembun said:
I just thought: maybe I should instead set up a jail for building for laptop, where source and object directories would be named /usr/src and /usr/obj? I've never worked with jails before, will it be a good place to start? Also, will I be able to export NFS /usr/src and /usr/obj from jail?
Click to expand...
Update: I did that and set up a classic thick jail for building sources for my laptop, where sources and objects are in the /usr/src and /usr/obj and shared these directories over NFS. Just compiled and installed the universe and everything works perfectly fine, (file names match naturally).
 
You must log in or register to reply here.
Back
Top