Installing FreeBSD manually (no installer)

ShelLuser

ShelLuser

Hi gang!

Editorial

SO yeah, at the time of writing I'm sitting behind my latest acquirement: a Dell PowerEdge SC1420, tower model. Its previous owner had no use for it anymore and so it ended up with me. Originally it ran several Windows versions and Ubuntu Linux but because I'm mostly familiar with 7 I re-arranged the whole thing until I had the first HD fully restored with Windows 7 (so: bootloader partition, Windows 7 system partition and a data partition). What to do with this thing?

I decided that it might make a nice server, and because of the extra resources I even wondered if I'd slap X onto it. Maybe a nice window / desktop manager? Alas... Because the original idea was to preserve Windows at first I figured I'd put FreeBSD onto the second HD, optionally grab Grub from the ports collection and go on from there.

And because I don't really care for the way the installer sets up a ZFS system I figured... Let's do things "different" :cool:

About this guide

This is a guide, but also a story about my new FreeBSD setup which I seriously enjoy so far (I'm typing this right now from within that same environment ;)). I'll show you how I did it, but I'm also trying to show you how I came to decide on certain things, or how I found the information I needed.

One live CD is all it takes

I grabbed disc1 from the repositories, booted into the live CD and really: that's all you need.

Partitioning

My server had everything set up with the MBR scheme so that was definitely open to improvement. I checked using sysctl kern.disks, learned about ada0, ada1, cd0 and cd1 and I was all set.

# gpart destroy -F ada1 (warning!) and my extra partition on disk 2 was toast. Now to set it all up for FreeBSD, ZFS powered: # gpart create -s gpt ada1 and I was now looking at a pristine space for my environment.

Step one is setting up a boot space; traditionally you need 1 boot partition and the rest can be data. If you ever wonder how large that partition needs to be then simply look in /boot: ls -l /boot/g*, this will show you the code which is going to be used. Either gptboot or gptzfsboot. At most 90kb in size so I decided that my first boot partition should be 128kb to be sure. Also note that gpart(8) is a very useful source of information here. There you'll learn that this partition should be smaller than 545kb.

# gpart add -t freebsd-boot -s 128k -l boot ada1, and we have a boot partition. Because I planned to use ZFS I didn't have to worry too much about file system sizes: we'll let ZFS worry about that. So the rest of this disk will be "ZFS only": # gpart add -t freebsd-zfs -l system ada1, and done.

Now for the actual filesystem. Being lazy as I am I start with using zfs import. It won't do much because we don't have anything available, but it will load the required ZFS drivers. I prefer doing this well before doing any actual writing. It is also important to note that some of the ZFS tools behavior changed over time; it now tries to auto-mount the pool name. You can overrule this, but why bother. This is why we're using a live cd, which has a write-able /tmp and /var. Convenient! First we make a new mountpoint: # mkdir /tmp/zfs, then we create the new ZFS pool:

# zpool create -m / -R /tmp/zfs zroot ada1p2. And while we're at it let's not forget to immediately mark this as our boot filesystem: # zpool set bootfs=zroot zroot. One of those silly things you might actually forget if you're not careful!

This will create our new pool called zroot which will be our first (and main) filesystem, bootable too. For now temporarily mounted on /tmp/zfs but in reality this is the new root system. Now its time to think about the filesystems you want to use.

Because it's no longer an issue of individual filesystems but more so about pools I prefer to keep / and /usr onto the same filesystem and work my way from there. Things I always do keep separated: /home, /var, /usr/local but also: /usr/src, /usr/ports and /usr/doc.

To be honest I take things even further than that ;) Now, I'm going to spoiler this thing because honestly: there is no "right" or "wrong" way here. I will share what I prefer to use, but seriously: study, experiment, and discover!

I prefer my system to be tidy, clean but also a little fool proof if it can be helped. With a touch of added security as well. So...

Code: 
# mkdir /tmp/zfs/usr
# zfs create zroot/home

# zfs create -o mountpoint=/usr/ports -o compression=on -o setuid=off zroot/ports
# zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/distfiles
# zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/packages

# zfs create -o mountpoint=/usr/local zroot/local
# zfs create -o mountpoint=/usr/src -o compression=on zroot/src
# zfs create -o mountpoint=/usr/doc -o compression=on zroot/doc

# zfs create zroot/var
# zfs create -o exec=off -o setuid=off zroot/var/db
# zfs create -o compression=on -o exec=on -o setuid=off zroot/var/db/pkg
# zfs create -o compression=on -o exec=off -o setuid=off zroot/var/mail
# zfs create -o compression=on -o exec=off -o setuid=off zroot/var/log
# zfs create -o exec=off -o setuid=off zroot/var/run
# zfs create -o exec=off -o setuid=off zroot/var/tmp
# zfs create -o exec=off -o setuid=off zroot/tmp
# chmod 1777 /tmp/zfs/tmp /tmp/zfs/var/tmp

# zfs create zroot/opt
# zfs create -o refquota=250M zroot/opt/jails
In case you're wondering... Why allow miscreants to execute stuff when there's no need? Most of the hack attempts I got during my days of hosting crappy websites for "skilled PHP coders" were triggered by 3rd parties trying to dump stuff into /tmp and from there trying to execute it. Good luck ;)

Also: ZFS has support for compression, so why not use it? Also considering that the sourcecode and documentation will mostly consist of (relatively easily) compressable ASCII.

Fair warning though: some build scripts out there are crappy. multimedia/ffmpeg is notorious for dumping stuff into /tmp and then whining when it can't execute things from there.

Seriously: don't you just love ZFS? Everything neatly separated (which is going to be more important in a bit) but none of your precious disk space will go to waste here!
ZFS = awesomeness:cool:

But... whatever you do you are going to need swap space, trust me: it can get nasty if you decide to skip that part. So: # zfs create -V 4G zroot/swap. You may need more, or you may need less (more likely), just adjust the size accordingly. What you do need is to mark this filesystem as swap: # zfs set org.freebsd:swap=on zroot/swap.

Installing the base system

Now we have the main hierarchy fully set up, so it's time to install the system:

Code: 
# cd /tmp/zfs
# tar xvJf /usr/freebsd-dist/base.txz
# tar xvJf /usr/freebsd-dist/kernel.txz
# tar xvJf /usr/freebsd-dist/lib32.txz
Although we are setting up our system to be fully bootable do note that it really isn't just yet. So if you want to you could chose to skip installing the kernel for now. Maybe you prefer rolling your own? But I do recommend against that: having a generic kernel around can be very useful. Just like /rescue is.

So now that we have our base system in place (only 650Mb or so) it's time to bootstrap it. Heck, there are CD's out there which will allow you to boot directly from a specific partition. And well.. Being prepared is half the battle! :p GI ... BSD! (sorry, though I wonder who will get this jest ;) ).

Bootstrap time: # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1.

However, we need to do more. Although we're not going to be booting this directly just yet I do want to make sure that we have all the basic settings right from the getgo. So:

Code: 
# cd /mnt/zfs/boot
# echo 'zfs_load="YES"' > loader.conf
# echo 'vfs.root.mountfrom="zfs:zroot"' >> loader.conf
# echo 'zfs_enable="YES"' > ../etc/rc.conf
Now, I used echo here to get my point across, in reality I simply fired up vi and got it over with.

So now we have a really pristine setup. It doesn't even know its own name nor in which timezone it resides. Poor lad (or lass) even lost its mind, you'll see. Now, we could do something about this right now, for example using chroot (its easier) but... Around this time the CD delays must be getting annoying, though I suppose it gets better over time (caching).

Don't feel bad if you need to use man a few times, that is one of the main reasons I booted into the live cd in the first place, so that we have a decent toolset at our disposal.

Either way.... Reboot! => # shutdown -r now.

You can export the ZFS filesystem if that makes you feel better, but... nah. I never bother, because this is honestly no different from rebooting your system (or shutting it down) whilst it is in operation.

Manual boot

Now things become tricky. Start the CD as before but as soon as you're in the menu hit escape in order to escape to the boot menu. It is time for more FreeBSD coolness :cool:

We're going to use the kernel (and some kernel modules) on the disc1 CD to boot our system. Start by using lsdev, this should show you an overview of the current disks and should display something like:

Code: 
disk1p1: FreeBSD boot
disk1p2: FreeBSD ZFS
Now we need to point the bootloader to our system, but also provide it with enough to, well, actually boot something.

Code: 
ok unload
ok load /boot/kernel/kernel
ok load /boot/kernel/opensolaris.ko
ok load /boot/kernel/zfs.ko
Start with this. This will load the kernel as well as the ZFS kernel modules. Yes, you can probably take it easy and just load zfs.ko which will auto-grab opensolaris. I have my reasons for doing it this way: what do you think is going to happen if your main kernel directory is trashed and you need to grab this from /boot/kernel.old/zfs.ko? You won't be auto loading anything, that's for sure. Doing it this way prepares you for that.

Sometimes its better not to take the easy way out, but to force yourself to go the full way. As a result of that I can now type this out from mind. So... something trashed my kernel? I no care ;)

But this is only step one. Now that we have prepared our system we need to tell it what to boot and from where. Remember that stuff we added to /boot/loader.conf? That wasn't for show you know ;)

Code: 
ok set currdev="disk1p2"
ok set vfs.root.mountfrom="zfs:zroot"
Now we're ready. If you feel daring then hit that magical command: boot.

Now.. don't get excited right away, all that white stuff? That would have happened either way, because that's merely the kernel doing its job. You know: the one we manually loaded?

The excitement starts when the whiteness ends (am I still allowed to say this in these politically correctness times? :D ).

Prepare for plenty of errors but also for plenty of automatically set up stuff. I wouldn't be surprised if your NIC's got auto discovered for example. And then... there is this:

Code: 
FreeBSD/amd64  (Amnesiac)  (ttyv0)

login:
Welcome... to the real world.

This is how you install FreeBSD without the installer. But hold on! We're not done yet. As you can see the poor thing doesn't even know its own name.

Log on using root and no password. And you might want to change that right away too.

First step: # tzsetup, and select your timezone.

Second step: fix your hostname, even if it's only temporary for now: # hostname mybox.network.mine (you might want to use something more realistic).

Third step: fix your e-mail. That's also why we fixed the hostname first. I know, everyone hates sendmail, but I don't. Besides, in these first stages it'll serve you well when ickyness happens. So:

Code: 
# cd /etc/mail
# make install
# service sendmail onerestart
There, doesn't that make you feel a lot better already? No? Oh well, tough crowd ;)

Next: # echo "hostname=`hostname`" >> /etc/rc.conf, this will preserve your current hostname.

And then we're pretty much done.

Next steps would be to set up your network by checking dmesg (or ifconfig) so that you can use that to set up your network. Let's say your NIC is called em0... You'd add something like this to /etc/rc.conf:

Code: 
ifconfig_em0="inet 192.168.10.5 netmask 255.255.255.0"
defaultrouter=192.168.10.1
Or maybe you want DHCP... Do keep in mind that if you're aiming for a manual IP address then you also need to set up /etc/resolv.conf:

Code: 
search network.mine
nameserver 8.8.8.8
If you are going with this route then don't think about rebooting just to "start your network", that's silly: # ifconfig em0 192.168.10.5 netmask 255.255.255.0 && route add default 192.168.10.1.

Now you're ready to grab the ports collection: # portsnap fetch extract, and from there on install all you need. Maybe Grub?

If you're going to be tweaking this setup be sure to get the source: # git clone https://git.freebsd.org/src.git /usr/src. I also recommend locking this filesystem afterwards so that no one can mess with it: # zfs set readonly=on zroot/src. You can even compile the system in this state without issues. Safety first!

Or get your hands on the source code for the documentation as well: # git clone https://git.freebsd.org/doc.git /usr/doc. Keep in mind that you'll need textproc/docproj to fully enjoy this one.

End notes

Some people wondered if you can somehow script the BSD installer to make it do stuff. Why?! I hope I made a point here, you don't need any of that. Make your own shell script, add the right commands and you're pretty much fully set to go. You don't need anything else.

Never forget that FreeBSD truly lives up the Unix philosophy. All that installer does... is starting many small things which, when combined together, make up for one bigger thing. But it still leaves those smaller individual parts open for you to use for yourself.
 
Last edited:
Great tutorial, thanks! :)
Just a question: is there a particular reason you choose to use UFS for the boot partition instead of ZFS (like the bootpool used by the installer)?
 
I've gone through it a couple of times but I fail to find any UFS partition. Or are you referring to the freebsd-boot partition? That actually doesn't have a filesystem at all. The MBR has just enough code to find the freebsd-boot partition on disk and load the contents into memory. See gptzfsboot(8).
 
SirDice said:
I've gone through it a couple of times but I fail to find any UFS partition. Or are you referring to the freebsd-boot partition? That actually doesn't have a filesystem at all. The MBR has just enough code to find the freebsd-boot partition on disk and load the contents into memory. See gptzfsboot(8).
Click to expand...
Yeah, exactly that. I tought it was an UFS partition. Thanks for the clarification and the hint. :)
 
getopt said:
The -J option is functional for creating archives only, so it makes little sense to advocate it in this context. :)
Click to expand...
Yeah, I'm well aware. The reason why I always use this parameter anyway is comparable to manually loading opensolaris.ko; I seldomly create xz archives and this helps me to remember that -J is used for these archive types.

I do the same with -z and -j though, but that's mostly force of habit.
 
Hi ShelLuser or somebody smart,

to modify the How-To for UEFI boot, I understand that what is needed is:

1. Replace gpart add -t freebsd-boot -s 128k -l boot ada1 with gpart add -t efi -s 200M -l efi ada1;
2. create a boot-pool gpart add -t freebsd-zfs -a 1m -s 2g -l boot-pool ada1 as a separate pool from the system pool. (This may not be necessary, just a preference).
3. Instead of loading the boot-code # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1 one has to load BOOTx64.efi to /efi/boot/BOOTx64.efi.

Does it look right? Can you advise on 3?

Kindest regards,

M

P.S. Figured it out and my script based on ShelLuser's How-To works.
 
Last edited:
  • Thanks
Reactions: veg
I could just hit the thanks button, but I want to personally thank you for this guide. I was wanting to do something special with the installer and was pulling my hair out trying to get it to do what I want. Using this guide I was able to build an installation script in pretty short order. My script installs a fully configured base system in about two minutes ready to take software packages.

You're absolutely right about going to a script over customizing the installer. If I had known how easy it was I would not have wasted time screwing around with it. I think it's pretty amazing the FreeBSD installation is that straightforward. The more I use this system the more I like it. It does everything in a simple and well thought out way.
 
Hi....
Thanks for your post:

I have a problem with installing freebsd manually by fully ZFS Encryption without passphrase on booting. I mean, I want to use key instead of a passphrase. Thanks in advance if you take a look at these commands and write me where do I have wrong instructions?
0.gpart create -s gpt ada0
1. gpart add -l swap0 -t freebsd-swap -a 1m -s 4G ada0
2.gpart add -t freebsd-boot -s 128k -l boot ada0
3.gpart add -t freebsd-zfs -l system ada0
4.mkdir -p /tmp/boot/key
5.dd if=/dev/random of=/tmp/boot/key/unbroken.key bs=1024 count=512
6.geli init -e aes -l 256 -s 4096 -P -K /tmp/boot/key/unbroken.key /dev/ada0p3
7.geli attach -p -k /tmp/boot/key/unbroken.key /dev/ada0p3
8.zpool create -m / -R /tmp/zfs zroot /dev/ada0p3.eli
9. zpool set bootfs=zroot zroot
10.zfs create -V 4G zroot/swap
11.zfs set org.freebsd:swap=on zroot/swap
12.cd /tmp/zfs
13.tar xvJf /usr/freebsd-dist/base.txz
14.tar xvJf /usr/freebsd-dist/kernel.txz
15.tar xvJf /usr/freebsd-dist/src.txz
16. tar xvJf /usr/freebsd-dist/ports.txz
17. gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
18. cp -r /tmp/boot/key /tmp/zfs/boot/
19. vi /tmp/zfs/boot/loader.conf
19.1 : zfs_load="YES"
19.2 : aesni_load="YES"
19:3 : geom_mirror_load="YES"
19.4 : geom_eli_load="yes"
19.5 : geli_ada0p3_keyfile0_load="YES"
19.6 : geli_ada0p3_keyfile0_type="ada0p3:geli_keyfile0"
19.7 : geli_ada0p3_keyfile0_name="/boot/key/unbroken.key"
20.vfs.root.mountfrom="zfs:zroot"
21. vi /tmp/zfs/etc/rc.conf
22. zfs_enable="YES"
23. printf "# Device\t\tMountpoint\tFStype\tOptions\t\tDump\tPass#\n" > /etc/fstab
24. printf "/dev/mirror/swap.eli\t\tnone\tswap\tsw\t\t0\t0\n" >> /etc/fstab
25. shutdown -r now

but when the system is rebooting, I've got this error :
No ZFS pools located, Can't boot

Thank you again for your consideration. :)
 
I haven't followed the developments recently but I'm pretty sure you still can't encrypt the partition holding the ZFS pool like that and boot directly from it, the reason being that the bootcode installed at step 17 just can't decrypt the pool on its own and load /boot/zfsloader from it.
 
kpa said:
I haven't followed the developments recently but I'm pretty sure you still can't encrypt the partition holding the ZFS pool like that and boot directly from it, the reason being that the bootcode installed at step 17 just can't decrypt the pool on its own and load /boot/zfsloader from it.
Click to expand...
You are right, but how can I do that? you mean first of all, decrypt partition while its booting?
 
Gave this a try last weekend, but I fear that my old CPU may become obsolete. Only difference being, I tried to use UFS, instead of ZFS; and that the .iso disks I used will not boot at all (no DRM/DRM2).

EDIT: More importantly no working boot-loader, either.
 
Last edited:
Thanks for this fine guide; I customized it to allow for setting up a dual boot between Arch Linux and FreeBSD on ZFS in a virtual machine. For details of this see this git repo of mine: https://github.com/fusion809/freebsd-zfs-manual-install.

Bash: 
zfs import
seems deprecated in 11.2, however, as it returns an error relating to not enough inputs. But it doesn't seem needed anyway as I think the ZFS drivers are automatically loaded in the live session now.
 
I am also thankful for this nice guide - Thanks ShelLuser, but I have a small problem:
After doing all the steps You described I am stuck at this point:
---
Code:
# cd /mnt/zfs/boot
# echo 'zfs_load="YES"' > loader.conf
# echo 'vfs.root.mountfrom="zfs:zroot"' >> loader.conf
# echo 'zfs_enable="YES"' > ../etc/rc.conf
---
I can not cd into this path /mnt/zfs/boot
it does not exist for me - maybe you can help me, what i might have done wrong
of cause I need to edit the loader.conf and rc.conf but cant :-(

Thanks in advance
 
chey said:
I can not cd into this path /mnt/zfs/boot
Click to expand...
When you created the pool, did you add the -R /tmp/zfs option? This causes the newly created pool to be mounted at /mnt/zfs. The boot/ directory gets created when you extract kernel.tgz.
 
Hello SirDice thanks for Your help, but I am still in the same trouble, here is everything I did:

Booted into livecd -> FreeBSD-11.2-RELEASE-amd64-disc1.iso

sysctl kern.disks
kern.disks: cd0 ada0
gpart destroy -F ada0
gpart create -s gpt ada0
gpart add -t freebsd-boot -s 128k -l boot ada0
gpart add -t freebsd-zfs -s 14G -l system ada0
mkdir /tmp/zfs
zpool create -f -m / -R /tmp/zfs zroot ada0p2
zpool set bootfs=zroot zroot

mkdir /tmp/zfs/usr
zfs create zroot/home
zfs create -o mountpoint=/usr/ports -o compression=on -o setuid=off zroot/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/packages

zfs create -o mountpoint=/usr/local zroot/local
zfs create -o mountpoint=/usr/src -o compression=on zroot/src
zfs create -o mountpoint=/usr/doc -o compression=on zroot/doc

zfs create zroot/var
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o compression=on -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o compression=on -o exec=off -o setuid=off zroot/var/mail
zfs create -o compression=on -o exec=off -o setuid=off zroot/var/log
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o exec=off -o setuid=off zroot/var/tmp
zfs create -o exec=off -o setuid=off zroot/tmp
chmod 1777 /tmp/zfs/tmp /tmp/zfs/var/tmp

zfs create zroot/opt
zfs create -o refquota=250M zroot/opt/jails

zfs create -V 4G zroot/swap
zfs set org.freebsd:swap=on zroot/swap

cd /tmp/zfs
tar xvJf /usr/freebsd-dist/base.txz
tar xvJf /usr/freebsd-dist/kernel.txz
tar xvJf /usr/freebsd-dist/lib32.txz

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

cd /mnt/zfs/boot

/mnt/zfs/boot: No such file or directory.

Sorry I don't know what I did wrong,
I thought the line: zpool create -f -m / -R /tmp/zfs zroot ada0p2

was okay, but it seems that its thing is not done correctly.
 
Hello fusion809 also thanks for Your script, at the end I found it:
# Bootstrap boot
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 4 ada0
cd /tmp/zfs/boot
echo 'zfs_load="YES"' > loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> loader.conf
echo 'zfs_enable="YES"' > ../etc/rc.conf

# Reboot time
shutdown -r now

I have to go to this path: /tmp/zfs/boot
I think that was my mistake.
 
Thanks for the brilliant writeup. I did something similar to triple boot and struggled to figure out what was going on. I eventually got my system working as desired, but didn’t really understand some of the magic. Your post really cleared up my understanding.
 
ShelLuser said:
Hi gang!

Editorial

SO yeah, at the time of writing I'm sitting behind my latest acquirement: a Dell PowerEdge SC1420, tower model. Its previous owner had no use for it anymore and so it ended up with me. Originally it ran several Windows versions and Ubuntu Linux but because I'm mostly familiar with 7 I re-arranged the whole thing until I had the first HD fully restored with Windows 7 (so: bootloader partition, Windows 7 system partition and a data partition). What to do with this thing?

I decided that it might make a nice server, and because of the extra resources I even wondered if I'd slap X onto it. Maybe a nice window / desktop manager? Alas... Because the original idea was to preserve Windows at first I figured I'd put FreeBSD onto the second HD, optionally grab Grub from the ports collection and go on from there.

And because I don't really care for the way the installer sets up a ZFS system I figured... Let's do things "different" :cool:

About this guide

This is a guide, but also a story about my new FreeBSD setup which I seriously enjoy so far (I'm typing this right now from within that same environment ;)). I'll show you how I did it, but I'm also trying to show you how I came to decide on certain things, or how I found the information I needed.

One live CD is all it takes

I grabbed disc1 from the repositories, booted into the live CD and really: that's all you need.

Partitioning

My server had everything set up with the MBR scheme so that was definitely open to improvement. I checked using sysctl kern.disks, learned about ada0, ada1, cd0 and cd1 and I was all set.

# gpart destroy -F ada1 (warning!) and my extra partition on disk 2 was toast. Now to set it all up for FreeBSD, ZFS powered: # gpart create -s gpt ada1 and I was now looking at a pristine space for my environment.

Step one is setting up a boot space; traditionally you need 1 boot partition and the rest can be data. If you ever wonder how large that partition needs to be then simply look in /boot: ls -l /boot/g*, this will show you the code which is going to be used. Either gptboot or gptzfsboot. At most 90kb in size so I decided that my first boot partition should be 128kb to be sure. Also note that gpart(8) is a very useful source of information here. There you'll learn that this partition should be smaller than 545kb.

# gpart add -t freebsd-boot -s 128k -l boot ada1, and we have a boot partition. Because I planned to use ZFS I didn't have to worry too much about file system sizes: we'll let ZFS worry about that. So the rest of this disk will be "ZFS only": # gpart add -t freebsd-zfs -l system ada1, and done.

Now for the actual filesystem. Being lazy as I am I start with using zfs import. It won't do much because we don't have anything available, but it will load the required ZFS drivers. I prefer doing this well before doing any actual writing. It is also important to note that some of the ZFS tools behavior changed over time; it now tries to auto-mount the pool name. You can overrule this, but why bother. This is why we're using a live cd, which has a write-able /tmp and /var. Convenient! First we make a new mountpoint: # mkdir /tmp/zfs, then we create the new ZFS pool:

# zpool create -m / -R /tmp/zfs zroot ada1p2. And while we're at it let's not forget to immediately mark this as our boot filesystem: # zpool set bootfs=zroot zroot. One of those silly things you might actually forget if you're not careful!

This will create our new pool called zroot which will be our first (and main) filesystem, bootable too. For now temporarily mounted on /tmp/zfs but in reality this is the new root system. Now its time to think about the filesystems you want to use.

Because it's no longer an issue of individual filesystems but more so about pools I prefer to keep / and /usr onto the same filesystem and work my way from there. Things I always do keep separated: /home, /var, /usr/local but also: /usr/src, /usr/ports and /usr/doc.

To be honest I take things even further than that ;) Now, I'm going to spoiler this thing because honestly: there is no "right" or "wrong" way here. I will share what I prefer to use, but seriously: study, experiment, and discover!

I prefer my system to be tidy, clean but also a little fool proof if it can be helped. With a touch of added security as well. So...

Code: 
# mkdir /tmp/zfs/usr
# zfs create zroot/home

# zfs create -o mountpoint=/usr/ports -o compression=on -o setuid=off zroot/ports
# zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/distfiles
# zfs create -o compression=off -o exec=off -o setuid=off zroot/ports/packages

# zfs create -o mountpoint=/usr/local zroot/local
# zfs create -o mountpoint=/usr/src -o compression=on zroot/src
# zfs create -o mountpoint=/usr/doc -o compression=on zroot/doc

# zfs create zroot/var
# zfs create -o exec=off -o setuid=off zroot/var/db
# zfs create -o compression=on -o exec=on -o setuid=off zroot/var/db/pkg
# zfs create -o compression=on -o exec=off -o setuid=off zroot/var/mail
# zfs create -o compression=on -o exec=off -o setuid=off zroot/var/log
# zfs create -o exec=off -o setuid=off zroot/var/run
# zfs create -o exec=off -o setuid=off zroot/var/tmp
# zfs create -o exec=off -o setuid=off zroot/tmp
# chmod 1777 /tmp/zfs/tmp /tmp/zfs/var/tmp

# zfs create zroot/opt
# zfs create -o refquota=250M zroot/opt/jails
In case you're wondering... Why allow miscreants to execute stuff when there's no need? Most of the hack attempts I got during my days of hosting crappy websites for "skilled PHP coders" were triggered by 3rd parties trying to dump stuff into /tmp and from there trying to execute it. Good luck ;)

Also: ZFS has support for compression, so why not use it? Also considering that the sourcecode and documentation will mostly consist of (relatively easily) compressable ASCII.

Fair warning though: some build scripts out there are crappy. multimedia/ffmpeg is notorious for dumping stuff into /tmp and then whining when it can't execute things from there.

Seriously: don't you just love ZFS? Everything neatly separated (which is going to be more important in a bit) but none of your precious disk space will go to waste here!
ZFS = awesomeness:cool:

But... whatever you do you are going to need swap space, trust me: it can get nasty if you decide to skip that part. So: # zfs create -V 4G zroot/swap. You may need more, or you may need less (more likely), just adjust the size accordingly. What you do need is to mark this filesystem as swap: # zfs set org.freebsd:swap=on zroot/swap.

Installing the base system

Now we have the main hierarchy fully set up, so it's time to install the system:

Code: 
# cd /tmp/zfs
# tar xvJf /usr/freebsd-dist/base.txz
# tar xvJf /usr/freebsd-dist/kernel.txz
# tar xvJf /usr/freebsd-dist/lib32.txz
Although we are setting up our system to be fully bootable do note that it really isn't just yet. So if you want to you could chose to skip installing the kernel for now. Maybe you prefer rolling your own? But I do recommend against that: having a generic kernel around can be very useful. Just like /rescue is.

So now that we have our base system in place (only 650Mb or so) it's time to bootstrap it. Heck, there are CD's out there which will allow you to boot directly from a specific partition. And well.. Being prepared is half the battle! :p GI ... BSD! (sorry, though I wonder who will get this jest ;) ).

Bootstrap time: # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1.

However, we need to do more. Although we're not going to be booting this directly just yet I do want to make sure that we have all the basic settings right from the getgo. So:

Code: 
# cd /mnt/zfs/boot
# echo 'zfs_load="YES"' > loader.conf
# echo 'vfs.root.mountfrom="zfs:zroot"' >> loader.conf
# echo 'zfs_enable="YES"' > ../etc/rc.conf
Now, I used echo here to get my point across, in reality I simply fired up vi and got it over with.

So now we have a really pristine setup. It doesn't even know its own name nor in which timezone it resides. Poor lad (or lass) even lost its mind, you'll see. Now, we could do something about this right now, for example using chroot (its easier) but... Around this time the CD delays must be getting annoying, though I suppose it gets better over time (caching).

Don't feel bad if you need to use man a few times, that is one of the main reasons I booted into the live cd in the first place, so that we have a decent toolset at our disposal.

Either way.... Reboot! => # shutdown -r now.

You can export the ZFS filesystem if that makes you feel better, but... nah. I never bother, because this is honestly no different from rebooting your system (or shutting it down) whilst it is in operation.

Manual boot

Now things become tricky. Start the CD as before but as soon as you're in the menu hit escape in order to escape to the boot menu. It is time for more FreeBSD coolness :cool:

We're going to use the kernel (and some kernel modules) on the disc1 CD to boot our system. Start by using lsdev, this should show you an overview of the current disks and should display something like:

Code: 
disk1p1: FreeBSD boot
disk1p2: FreeBSD ZFS
Now we need to point the bootloader to our system, but also provide it with enough to, well, actually boot something.

Code: 
ok unload
ok load /boot/kernel/kernel
ok load /boot/kernel/opensolaris.ko
ok load /boot/kernel/zfs.ko
Start with this. This will load the kernel as well as the ZFS kernel modules. Yes, you can probably take it easy and just load zfs.ko which will auto-grab opensolaris. I have my reasons for doing it this way: what do you think is going to happen if your main kernel directory is trashed and you need to grab this from /boot/kernel.old/zfs.ko? You won't be auto loading anything, that's for sure. Doing it this way prepares you for that.

Sometimes its better not to take the easy way out, but to force yourself to go the full way. As a result of that I can now type this out from mind. So... something trashed my kernel? I no care ;)

But this is only step one. Now that we have prepared our system we need to tell it what to boot and from where. Remember that stuff we added to /boot/loader.conf? That wasn't for show you know ;)

Code: 
ok set currdev="disk1p2"
ok set vfs.root.mountfrom="zfs:zroot"
Now we're ready. If you feel daring then hit that magical command: boot.

Now.. don't get excited right away, all that white stuff? That would have happened either way, because that's merely the kernel doing its job. You know: the one we manually loaded?

The excitement starts when the whiteness ends (am I still allowed to say this in these politically correctness times? :D ).

Prepare for plenty of errors but also for plenty of automatically set up stuff. I wouldn't be surprised if your NIC's got auto discovered for example. And then... there is this:

Code: 
FreeBSD/amd64  (Amnesiac)  (ttyv0)

login:
Welcome... to the real world.

This is how you install FreeBSD without the installer. But hold on! We're not done yet. As you can see the poor thing doesn't even know its own name.

Log on using root and no password. And you might want to change that right away too.

First step: # tzsetup, and select your timezone.

Second step: fix your hostname, even if it's only temporary for now: # hostname mybox.network.mine (you might want to use something more realistic).

Third step: fix your e-mail. That's also why we fixed the hostname first. I know, everyone hates sendmail, but I don't. Besides, in these first stages it'll serve you well when ickyness happens. So:

Code: 
# cd /etc/mail
# make install
# service sendmail onerestart
There, doesn't that make you feel a lot better already? No? Oh well, tough crowd ;)

Next: # echo "hostname=`hostname`" >> /etc/rc.conf, this will preserve your current hostname.

And then we're pretty much done.

Next steps would be to set up your network by checking dmesg (or ifconfig) so that you can use that to set up your network. Let's say your NIC is called em0... You'd add something like this to /etc/rc.conf:

Code: 
ifconfig_em0="inet 192.168.10.5 netmask 255.255.255.0"
defaultrouter=192.168.10.1
Or maybe you want DHCP... Do keep in mind that if you're aiming for a manual IP address then you also need to set up /etc/resolv.conf:

Code: 
search network.mine
nameserver 8.8.8.8
If you are going with this route then don't think about rebooting just to "start your network", that's silly: # ifconfig em0 192.168.10.5 netmask 255.255.255.0 && route add default 192.168.10.1.

Now you're ready to grab the ports collection: # portsnap fetch extract, and from there on install all you need. Maybe Grub?

If you're going to be tweaking this setup be sure to get the source: # svnlite co https://svn.freebsd.org/base/releng/11.1 /usr/src. I also recommend locking this filesystem afterwards so that no one can mess with it: # zfs set readonly=on zroot/src. You can even compile the system in this state without issues. Safety first!

Or get your hands on the source code for the documentation as well: # svnlite co https://svn.freebsd.org/doc/head /usr/doc. Keep in mind that you'll need textproc/docproj to fully enjoy this one.

End notes

Some people wondered if you can somehow script the BSD installer to make it do stuff. Why?! I hope I made a point here, you don't need any of that. Make your own shell script, add the right commands and you're pretty much fully set to go. You don't need anything else.

Never forget that FreeBSD truly lives up the Unix philosophy. All that installer does... is starting many small things which, when combined together, make up for one bigger thing. But it still leaves those smaller individual parts open for you to use for yourself.
Click to expand...
You can alternatively do it in one (no warranty) command line:
sh deploy-freebsd.sh

https://github.com/spartrekus/deploy-FreeBSD/blob/master/deploy-freebsd.sh

Your method looks in similar direction, with unpacking the content of archives.

;)
 
I used this guide this weekend to install FreeBSD on a hard drive that already had a copy of Linux(Debian) installed. My intent was to dual-boot Linux and FreeBSD, so I had to make a few adjustments.

I was using the FreeBSD 12-Release ISO disc 1.

1. Skip the commands where gpart is used to delete and re-create the partition table.
2. Skip creation of a boot partition. I already have a ESP set up for UEFI booting. (with rEFInd as boot manager)
2. Skip steps related to installing anything to the MBR or the boot partition.
3. For the step where the base.txz and kernel.txz files are untarred, with the commands,
Code: 
# cd /tmp/zfs
# tar xvJf /usr/freebsd-dist/base.txz
# tar xvJf /usr/freebsd-dist/kernel.txz
# tar xvJf /usr/freebsd-dist/lib32.txz
That didn't work for me. The /usr/freebsd-dist directory contained only one file, named MANIFEST. That file (MANIFEST) has references to the base.txz and kernel.txz files, but that didn't help me find the actual files.
I found another guide which suggested running fetch commands to download the necessary files from FreeBSD ftp sites. I wasn't successful with that approach either, as it seemed I didn't have the network configured sufficiently at that point of the install to be able to run the fetch commands. I got around that by using a different computer to download the three txz files onto a USB flash drive, then mounting the flash drive on the target computer and mounting it with
Code: 
# mkdir /mnt/usbdisk
# mount -t msdosfs  /dev/da1p1 /mnt/usbdisk

For the part that describes creating the loader.conf and rc.conf files, I encountered the same problem as described by another user in post # 14. The solution was noting a typo in the instructions. It says to first run
Code: 
# cd /mnt/zfs
but it should be
Code: 
# cd /tmp/zfs
.
Finally, I had to run the following to get REFInd to boot the new FreeBSD system...
Code: 
# mkdir /tmp/esp
# mount -t msdosfs /dev/ada0p1 /tmp/esp
# cd /tmp/esp/EFI
# mkdir FreeBSD
# cp /boot/loader.efi /tmp/esp/EFI/FreeBSD/
.
I do have a question: Does anyone know where the base.txz, kernel.txz and lib32.txz files can be found on the install iso disc? They must exist on it somewhere :). I ran the find command to try to locate them, but without success.

Thanks for ShelLuser for this guide!
 
You must log in or register to reply here.
Back
Top