cross compiling world for i386 on amd64 platform

Beeblebrox

Beeblebrox

I have 2 laptops with P3 chips which I would like to use for testing on FreeBSD. The requirements I have established for this test are:
A. Laptop #1 will boot as diskless through pxe, so will need a i386 kernel and will run X11 from amd64 server.
B. Laptop #2 will have i386 custom (same as #1) kernel, world & various ports installed.
C. All ports for i386 will be cross-compiled on the amd64 system, and placed into repositories on amd64 as i386distfiles & i386pkg.

The steps to do all this as far as I can tell:
1: Prepare i386KRNCFG and place in /usr/src/sys/i386/conf
2: [cmd=]# cd /usr/src[/cmd]
[cmd=]# env MACHINE_ARCH=amd64 MACHINE=amd64[/cmd]
[cmd=]# env DESTDIR=/diskless/i386[/cmd]
[cmd=]# make TARGET=i386 buildworld[/CMD]
3: [cmd=]# make TARGET=i386 KERNCONF=i386KRNCFG buildkernel[/cmd]

4-a: [cmd=]# make installworld[/cmd]
4-b: [cmd=]# make installkernel KERNCONF=i386KRNCFG[/cmd]
This is all that is necessary for section A above; the pxe environemnt is now ready. As I can tell, an i386 world will need to be preserved on the amd64 system, which is under /diskless/i386.

5: immediately after step 4 and staying in the same env:
[CMD=]# cd /diskless/i386/etc && make distribution[/CMD]

6: [CMD=]# mergemaster -A i386 -D /diskless/i386[/CMD]

MY QUESTIONS:
i- Are the steps listed correct - any mistakes? Do I need to set a different MAKEOBJDIRPREFIX for the i386 build?
ii- In step 5, where will the iso be placed - is there a dir= switch?
iii- Since an i386 environemnt will always be present on the amd64 system, how should I switch into it when I want to build ports? chroot or other mechanism?
iv- How should I specify the PKGREPOSITORY, WRKDIRPREFIX etc, as well as other make.conf variables? My solution is to modify /diskless/i386/etc/make.conf then chroot to /diskless/i386 and run
# make package-recursive -C /usr/ports/net/foo
This command will install the package however and don't know how to prevent that.
v- Do ports which have already been installed on the amd64 server need to be re-installed into the i386 (Laptop #1) environment? I assume yes, since the userland programs are different for i386 vs amd64.
vi- When making build world & kernel, is there any modification / flag which must be passed to the make.conf on the amd64 env?

Long list, but I hope others can benefit from this in the future, since info on this topic seems a little scarce and scattered...
 
Not entirely related to your questions, but I do have one machine, used for diskless booting various clients, that is running amd64 8-stable (8.2 now).

It has few jails, that run (and build inside the jails)

7-stable i386
7-stable amd64
8-stable i386
8-stable amd64
9-current amd64
(didn't think of 9-current i386 yet)

So far, no problems. Within each jail, I run the usual buildworld/buildkernel targets and rebuild all packages.

I guess you can do something like that as well.
 
Thanks, 3 questions:

1. How much memory overhead does each jail & client/jail place on resources?
2. How do you manage /home/user settings, or is each user static with the version & architecture he logs in with?
3. This post is related. How do you handle the problem, or do you not have any workstations / laptops?
 
The jail takes virtually no memory when it runs. It does not have to run however, except at times you want to build things inside the jail, that is, when you want to access the client environment. If you chose to leave the jail running all the time, you typically have syslogd, cron and not much more. Perhaps sshd, if you need remote access.

The user's home directories are usually separate from the boot environment. In my case, these reside on different server(s) from the boot server. For each particular client workstation, you can define pretty much any aspect of it's boot environment from one central location.

Most of my diskless clients are X workstations. There are however all kind of others, including servers and even some embedded systems. The ability to boot from different OS versions sometimes is very helpful. Especially on troublesome/new hardware.

I have commented on your other thread.
 
@danbi,

I have the impression that the kernel is always shared among jails. How are you able to maintain different versions for each jail ?

Thanks
 
Yes, the host kernel is the same. All the host kernel needs to do is to be able to run the binaries of various versions you run within jails. They have their own versions of libraries within the jails, so that does not concern the host. I imagine, if at some point FreeBSD 9 or 10 binaries (executable format) become incompatible, that would require upgrade of the host's version. New kernels are supposed to execute old version binaries. Of course, I am not considering the various syscalls, such as networking --- my jails are used only for compiling and thankfully the compiler is pretty much portable.

Here is an example:

Code: 
(the host 8-stable amd64)
# cc -v
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070719  [FreeBSD]

(hosted jails)
# jls
   JID  IP Address      Hostname                      Path
     1  xxx.xxx.xxx.xxx xxx.xxx.xxx                   /jails/freebsd7-32
     2  xxx.xxx.xxx.xxx xxx.xxx.xxx                   /jails/freebsd8-32
     3  xxx.xxx.xxx.xxx xxx.xxx.xxx                   /jails/freebsd8-64
     4  xxx.xxx.xxx.xxx xxx.xxx.xxx                   /jails/freebsd9-64

(jail 1 7-stable i386)
# jexec 1 csh
# cc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070719  [FreeBSD]

(jail 2 8-stable i386)
# jexec 2 csh
# cc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.1 20070719  [FreeBSD]

(jail 3 8-stable amd64 -- same as the host)
# jexec 3 csh
# cc -v
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070719  [FreeBSD]

(jail 4 9-current amd64)
# jexec 4 csh
# cc -v
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.2 20070831 prerelease [FreeBSD]

In each of the jails I do keep it's own source tree, mount a common ports tree and compile world/kernel/ports.

About the only issue I have so far is with 9-current's csup, sticking in 'uwait' -- I have to populate it's /usr/src from 'outside' for now. Haven't time to figure out why it happens. Perhaps syscall incompatibility. I know it will be resolved by upgrading the host to 9-current, but am still lazy :)

Perhaps I would get into trouble if the jails are used for things much different than compiling code inside.
The purpose of this system is limited to serving diskless clients and compiling software for these, which it does nicely.
 
You must log in or register to reply here.
Back
Top