• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Installing FreeBSD manually (no installer)

ShelLuser

Daemon

Thanks: 1,098
Best answers: 2
Messages: 2,426

#1
Hi gang!

Editorial

SO yeah, at the time of writing I'm sitting behind my latest acquirement: a Dell PowerEdge SC1420, tower model. Its previous owner had no use for it anymore and so it ended up with me. Originally it ran several Windows versions and Ubuntu Linux but because I'm mostly familiar with 7 I re-arranged the whole thing until I had the first HD fully restored with Windows 7 (so: bootloader partition, Windows 7 system partition and a data partition). What to do with this thing?

I decided that it might make a nice server, and because of the extra resources I even wondered if I'd slap X onto it. Maybe a nice window / desktop manager? Alas... Because the original idea was to preserve Windows at first I figured I'd put FreeBSD onto the second HD, optionally grab Grub from the ports collection and go on from there.

And because I don't really care for the way the installer sets up a ZFS system I figured... Let's do things "different" :cool:

About this guide

This is a guide, but also a story about my new FreeBSD setup which I seriously enjoy so far (I'm typing this right now from within that same environment ;)). I'll show you how I did it, but I'm also trying to show you how I came to decide on certain things, or how I found the information I needed.

One live CD is all it takes

I grabbed disc1 from the repositories, booted into the live CD and really: that's all you need.

Partitioning

My server had everything set up with the MBR scheme so that was definitely open to improvement. I checked using sysctl kern.disks, learned about ada0, ada1, cd0 and cd1 and I was all set.

# gpart destroy -F ada1 (warning!) and my extra partition on disk 2 was toast. Now to set it all up for FreeBSD, ZFS powered: # gpart create -s gpt ada1 and I was now looking at a pristine space for my environment.

Step one is setting up a boot space; traditionally you need 1 boot partition and the rest can be data. If you ever wonder how large that partition needs to be then simply look in /boot: ls -l /boot/g*, this will show you the code which is going to be used. Either gptboot or gptzfsboot. At most 90kb in size so I decided that my first boot partition should be 128kb to be sure. Also note that gpart(8) is a very useful source of information here. There you'll learn that this partition should be smaller than 545kb.

# gpart add -t freebsd-boot -s 128k -l boot ada1, and we have a boot partition. Because I planned to use ZFS I didn't have to worry too much about file system sizes: we'll let ZFS worry about that. So the rest of this disk will be "ZFS only": # gpart add -t freebsd-zfs -l system ada1, and done.

Now for the actual filesystem. Being lazy as I am I start with using zfs import. It won't do much because we don't have anything available, but it will load the required ZFS drivers. I prefer doing this well before doing any actual writing. It is also important to note that some of the ZFS tools behavior changed over time; it now tries to auto-mount the pool name. You can overrule this, but why bother. This is why we're using a live cd, which has a write-able /tmp and /var. Convenient! First we make a new mountpoint: # mkdir /tmp/zfs, then we create the new ZFS pool:

# zpool create -m / -R /tmp/zfs zroot ada1p2. And while we're at it let's not forget to immediately mark this as our boot filesystem: # zpool set bootfs=zroot zroot. One of those silly things you might actually forget if you're not careful!

This will create our new pool called zroot which will be our first (and main) filesystem, bootable too. For now temporarily mounted on /tmp/zfs but in reality this is the new root system. Now its time to think about the filesystems you want to use.

Because it's no longer an issue of individual filesystems but more so about pools I prefer to keep / and /usr onto the same filesystem and work my way from there. Things I always do keep separated: /home, /var, /usr/local but also: /usr/src, /usr/ports and /usr/doc.

To be honest I take things even further than that ;) Now, I'm going to spoiler this thing because honestly: there is no "right" or "wrong" way here. I will share what I prefer to use, but seriously: study, experiment, and discover!

I prefer my system to be tidy, clean but also a little fool proof if it can be helped. With a touch of added security as well. So...

Code:
# mkdir /tmp/zfs/usr
# zfs create zroot/home

# zfs create -o mountpoint=/usr/ports -o compression=on -o setuid=off zroot/ports
# zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/distfiles
# zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/packages

# zfs create -o mountpoint=/usr/local zroot/local
# zfs create -o mountpoint=/usr/src -o compression=on zroot/src
# zfs create -o mountpoint=/usr/doc -o compression=on zroot/doc

# zfs create zroot/var
# zfs create -o exec=off -o setuid=off zroot/var/db
# zfs create -o compression=on -o exec=on -o setuid=off zroot/var/db/pkg
# zfs create -o compression=on -o exec=off -o setuid=off zroot/var/mail
# zfs create -o compression=on -o exec=off -o setuid=off zroot/var/log
# zfs create -o exec=off -o setuid=off zroot/var/run
# zfs create -o exec=off -o setuid=off zroot/var/tmp
# zfs create -o exec=off -o setuid=off zroot/tmp
# chmod 1777 /tmp/zfs/tmp /tmp/zfs/var/tmp

# zfs create zroot/opt
# zfs create -o refquota=250M zroot/opt/jails
In case you're wondering... Why allow miscreants to execute stuff when there's no need? Most of the hack attempts I got during my days of hosting crappy websites for "skilled PHP coders" were triggered by 3rd parties trying to dump stuff into /tmp and from there trying to execute it. Good luck ;)

Also: ZFS has support for compression, so why not use it? Also considering that the sourcecode and documentation will mostly consist of (relatively easily) compressable ASCII.

Fair warning though: some build scripts out there are crappy. multimedia/ffmpeg is notorious for dumping stuff into /tmp and then whining when it can't execute things from there.

Seriously: don't you just love ZFS? Everything neatly separated (which is going to be more important in a bit) but none of your precious disk space will go to waste here!
ZFS = awesomeness :cool:

But... whatever you do you are going to need swap space, trust me: it can get nasty if you decide to skip that part. So: # zfs create -V 4G zroot/swap. You may need more, or you may need less (more likely), just adjust the size accordingly. What you do need is to mark this filesystem as swap: # zfs set org.freebsd:swap=on zroot/swap.

Installing the base system

Now we have the main hierarchy fully set up, so it's time to install the system:

Code:
# cd /tmp/zfs
# tar xvJf /usr/freebsd-dist/base.txz
# tar xvJf /usr/freebsd-dist/kernel.txz
# tar xvJf /usr/freebsd-dist/lib32.txz
Although we are setting up our system to be fully bootable do note that it really isn't just yet. So if you want to you could chose to skip installing the kernel for now. Maybe you prefer rolling your own? But I do recommend against that: having a generic kernel around can be very useful. Just like /rescue is.

So now that we have our base system in place (only 650Mb or so) it's time to bootstrap it. Heck, there are CD's out there which will allow you to boot directly from a specific partition. And well.. Being prepared is half the battle! :p GI ... BSD! (sorry, though I wonder who will get this jest ;) ).

Bootstrap time: # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1.

However, we need to do more. Although we're not going to be booting this directly just yet I do want to make sure that we have all the basic settings right from the getgo. So:

Code:
# cd /mnt/zfs/boot
# echo 'zfs_load="YES"' > loader.conf
# echo 'vfs.root.mountfrom="zfs:zroot"' >> loader.conf
# echo 'zfs_enable="YES"' > ../etc/rc.conf
Now, I used echo here to get my point across, in reality I simply fired up vi and got it over with.

So now we have a really pristine setup. It doesn't even know its own name nor in which timezone it resides. Poor lad (or lass) even lost its mind, you'll see. Now, we could do something about this right now, for example using chroot (its easier) but... Around this time the CD delays must be getting annoying, though I suppose it gets better over time (caching).

Don't feel bad if you need to use man a few times, that is one of the main reasons I booted into the live cd in the first place, so that we have a decent toolset at our disposal.

Either way.... Reboot! => # shutdown -r now.

You can export the ZFS filesystem if that makes you feel better, but... nah. I never bother, because this is honestly no different from rebooting your system (or shutting it down) whilst it is in operation.

Manual boot

Now things become tricky. Start the CD as before but as soon as you're in the menu hit escape in order to escape to the boot menu. It is time for more FreeBSD coolness :cool:

We're going to use the kernel (and some kernel modules) on the disc1 CD to boot our system. Start by using lsdev, this should show you an overview of the current disks and should display something like:

Code:
disk1p1: FreeBSD boot
disk1p2: FreeBSD ZFS
Now we need to point the bootloader to our system, but also provide it with enough to, well, actually boot something.

Code:
[FILE]ok[/FILE] unload
[FILE]ok[/FILE] load /boot/kernel/kernel
[FILE]ok[/FILE] load /boot/kernel/opensolaris.ko
[FILE]ok[/FILE] load /boot/kernel/zfs.ko
Start with this. This will load the kernel as well as the ZFS kernel modules. Yes, you can probably take it easy and just load zfs.ko which will auto-grab opensolaris. I have my reasons for doing it this way: what do you think is going to happen if your main kernel directory is trashed and you need to grab this from /boot/kernel.old/zfs.ko? You won't be auto loading anything, that's for sure. Doing it this way prepares you for that.

Sometimes its better not to take the easy way out, but to force yourself to go the full way. As a result of that I can now type this out from mind. So... something trashed my kernel? I no care ;)

But this is only step one. Now that we have prepared our system we need to tell it what to boot and from where. Remember that stuff we added to /boot/loader.conf? That wasn't for show you know ;)

Code:
[FILE]ok[/FILE] set currdev="disk1p2"
[FILE]ok[/FILE] set vfs.root.mountfrom="zfs:zroot"
Now we're ready. If you feel daring then hit that magical command: boot.

Now.. don't get excited right away, all that white stuff? That would have happened either way, because that's merely the kernel doing its job. You know: the one we manually loaded?

The excitement starts when the whiteness ends (am I still allowed to say this in these politically correctness times? :D ).

Prepare for plenty of errors but also for plenty of automatically set up stuff. I wouldn't be surprised if your NIC's got auto discovered for example. And then... there is this:

Code:
FreeBSD/amd64  (Amnesiac)  (ttyv0)

login:
Welcome... to the real world.

This is how you install FreeBSD without the installer. But hold on! We're not done yet. As you can see the poor thing doesn't even know its own name.

Log on using root and no password. And you might want to change that right away too.

First step: # tzsetup, and select your timezone.

Second step: fix your hostname, even if it's only temporary for now: # hostname mybox.network.mine (you might want to use something more realistic).

Third step: fix your e-mail. That's also why we fixed the hostname first. I know, everyone hates sendmail, but I don't. Besides, in these first stages it'll serve you well when ickyness happens. So:

Code:
# cd /etc/mail
# make install
# service sendmail onerestart
There, doesn't that make you feel a lot better already? No? Oh well, tough crowd ;)

Next: # echo "hostname=`hostname`" >> /etc/rc.conf, this will preserve your current hostname.

And then we're pretty much done.

Next steps would be to set up your network by checking dmesg (or ifconfig) so that you can use that to set up your network. Let's say your NIC is called em0... You'd add something like this to /etc/rc.conf:

Code:
ifconfig_em0="inet 192.168.10.5 netmask 255.255.255.0"
defaultrouter=192.168.10.1
Or maybe you want DHCP... Do keep in mind that if you're aiming for a manual IP address then you also need to set up /etc/resolv.conf:

Code:
search network.mine
nameserver 8.8.8.8
If you are going with this route then don't think about rebooting just to "start your network", that's silly: # ifconfig em0 192.168.10.5 netmask 255.255.255.0 && route add default 192.168.10.1.

Now you're ready to grab the ports collection: # portsnap fetch extract, and from there on install all you need. Maybe Grub?

If you're going to be tweaking this setup be sure to get the source: # svnlite co https://svn.freebsd.org/base/releng/11.1 /usr/src. I also recommend locking this filesystem afterwards so that no one can mess with it: # zfs set readonly=on zroot/src. You can even compile the system in this state without issues. Safety first!

Or get your hands on the source code for the documentation as well: # svnlite co https://svn.freebsd.org/doc/head /usr/doc. Keep in mind that you'll need textproc/docproj to fully enjoy this one.

End notes

Some people wondered if you can somehow script the BSD installer to make it do stuff. Why?! I hope I made a point here, you don't need any of that. Make your own shell script, add the right commands and you're pretty much fully set to go. You don't need anything else.

Never forget that FreeBSD truly lives up the Unix philosophy. All that installer does... is starting many small things which, when combined together, make up for one bigger thing. But it still leaves those smaller individual parts open for you to use for yourself.
 
Last edited:

fullauto2012

Active Member

Thanks: 27
Messages: 165

#2
This was really great... I can't wait to try this for myself.
 

Maxnix

Well-Known Member

Thanks: 159
Messages: 279

#3
Great tutorial, thanks! :)
Just a question: is there a particular reason you choose to use UFS for the boot partition instead of ZFS (like the bootpool used by the installer)?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Thanks: 5,891
Best answers: 7
Messages: 26,510

#4
I've gone through it a couple of times but I fail to find any UFS partition. Or are you referring to the freebsd-boot partition? That actually doesn't have a filesystem at all. The MBR has just enough code to find the freebsd-boot partition on disk and load the contents into memory. See gptzfsboot(8).
 

Maxnix

Well-Known Member

Thanks: 159
Messages: 279

#5
I've gone through it a couple of times but I fail to find any UFS partition. Or are you referring to the freebsd-boot partition? That actually doesn't have a filesystem at all. The MBR has just enough code to find the freebsd-boot partition on disk and load the contents into memory. See gptzfsboot(8).
Yeah, exactly that. I tought it was an UFS partition. Thanks for the clarification and the hint. :)
 

ShelLuser

Daemon

Thanks: 1,098
Best answers: 2
Messages: 2,426

#6
The -J option is functional for creating archives only, so it makes little sense to advocate it in this context. :)
Yeah, I'm well aware. The reason why I always use this parameter anyway is comparable to manually loading opensolaris.ko; I seldomly create xz archives and this helps me to remember that -J is used for these archive types.

I do the same with -z and -j though, but that's mostly force of habit.
 

mefizto

Well-Known Member

Thanks: 5
Messages: 354

#7
Hi ShelLuser or somebody smart,

to modify the How-To for UEFI boot, I understand that what is needed is:

1. Replace gpart add -t freebsd-boot -s 128k -l boot ada1 with gpart add -t efi -s 200M -l efi ada1;
2. create a boot-pool gpart add -t freebsd-zfs -a 1m -s 2g -l boot-pool ada1 as a separate pool from the system pool. (This may not be necessary, just a preference).
3. Instead of loading the boot-code # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1 one has to load BOOTx64.efi to /efi/boot/BOOTx64.efi.

Does it look right? Can you advise on 3?

Kindest regards,

M

P.S. Figured it out and my script based on ShelLuser's How-To works.
 
Last edited:

CraigHB

Member

Thanks: 25
Messages: 95

#8
I could just hit the thanks button, but I want to personally thank you for this guide. I was wanting to do something special with the installer and was pulling my hair out trying to get it to do what I want. Using this guide I was able to build an installation script in pretty short order. My script installs a fully configured base system in about two minutes ready to take software packages.

You're absolutely right about going to a script over customizing the installer. If I had known how easy it was I would not have wasted time screwing around with it. I think it's pretty amazing the FreeBSD installation is that straightforward. The more I use this system the more I like it. It does everything in a simple and well thought out way.
 
Top