HOWTO: Install FreeBSD 9.0 RELEASE (Root on UFS + ZFS, RAID1)

I'll show how to install FreeBSD 9.0 RELEASE on the machine with 2 disks (RAID 1). I'll use the following logical layout:

Code:
UFS 1G   /           ro
ZFS *    /home       rw
ZFS *    /tmp        rw
UFS *    /usr        ro | softupdates (mounted r/w only for packages updates)
ZFS *    /usr/obj    rw 
ZFS *    /usr/ports  rw 
ZFS *    /usr/src    rw 
ZFS *    /var        rw
UFS 1G   /var/db/pkg ro | softupdates (mounted r/w only for packages updates)
 
(1) Boot from a FreeBSD9 installation DVD or memstick and choose "Live CD".

(2) Load the necessary kernel modules.
Code:
# kldload geom_mbr.ko
# kldload geom_mirror.ko
# kldload opensolaris.ko
# kldload zfs.ko

(3) Create the necessary partitions on the disks.
Code:
# sh
# disk0=da0
# disk1=da1
# gpart create -s gpt $disk0
# gpart add -b 34 -s 94 -t freebsd-boot $disk0
# gpart add -s 1G -t freebsd-ufs $disk0
# gpart add -s 4G -t freebsd-swap $disk0
# gpart add -s 1G -t freebsd-ufs $disk0
# gpart add -s 8G -t freebsd-ufs $disk0
# gpart add -t freebsd-zfs $disk0
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 $disk0
# gpart backup $disk0 | gpart restore -F $disk1

(4) Create RAID1 for UFS and swap filesystems.
Code:
# gmirror label  root  ${disk0}p2 ${disk1}p2
# gmirror label  swap  ${disk0}p3 ${disk1}p3
# gmirror label  pkg   ${disk0}p4 ${disk1}p4
# gmirror label  usr   ${disk0}p5 ${disk1}p5
 
(5) Create filesystems and mount them.
Code:
# newfs -m 1 /dev/mirror/root
# newfs -m 1 -U /dev/mirror/pkg
# newfs -m 1 -U /dev/mirror/usr
# mkdir /tmp/NEWROOT
# mount /dev/mirror/root /tmp/NEWROOT
# mkdir /tmp/NEWROOT/usr
# mount /dev/mirror/usr /tmp/NEWROOT/usr
# zpool create -m /tmp/NEWROOT/basefs basefs mirror ${disk0}p6 ${disk1}p6
# zfs create -o mountpoint=/tmp/NEWROOT/home basefs/home
# zfs create -o mountpoint=/tmp/NEWROOT/var basefs/var
# zfs create -o mountpoint=/tmp/NEWROOT/tmp basefs/tmp
# zfs create -o mountpoint=none basefs/usr
# zfs create -o mountpoint=/tmp/NEWROOT/usr/obj basefs/usr/obj
# zfs create -o mountpoint=/tmp/NEWROOT/usr/ports basefs/usr/ports
# zfs create -o mountpoint=/tmp/NEWROOT/usr/src basefs/usr/src

P.S. Feel free to play with ZFS filesystems options and create additional filesystems if you need/want.

(6) Install FreeBSD.
Code:
# cd /usr/freebsd-dist
# export DESTDIR=/tmp/NEWROOT
# for file in base.txz lib32.txz kernel.txz;
# do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
 
(7) Provide basic configuration.
Code:
# cat > /tmp/NEWROOT/etc/fstab << __EOF__             
#dev                #mount        #fs  #opts #dump #pass
/dev/mirror/root    /             ufs  rw    1     1     
/dev/mirror/swap    none          swap sw    0     0  
/dev/mirror/usr     /usr          ufs  rw    2     2  
basefs/var          /var          zfs  rw    0     0
/dev/mirror/pkg     /var/db/pkg   ufs  rw    2     2  

basefs              /basefs       zfs  rw    0     0
basefs/home         /home         zfs  rw    0     0
basefs/usr/obj      /usr/obj      zfs  rw    0     0
basefs/usr/ports    /usr/ports    zfs  rw    0     0
basefs/usr/src      /usr/src      zfs  rw    0     0
basefs/tmp          /tmp          zfs  rw    0     0
__EOF__   

# cat > /tmp/NEWROOT/boot/loader.conf << __EOF__      
zfs_load="YES"                                  
ahci_load="YES"                                 
geom_mirror_load="YES"                          
__EOF__  

# cat > /tmp/NEWROOT/etc/rc.conf << __EOF__
zfs_enable="YES"
__EOF__

(8) Unmount filesystems and reboot.
Code:
# cd /
# zfs umount -a
# umount /tmp/NEWROOT/usr
# umount /tmp/NEWROOT
# zfs set mountpoint=/home basefs/home
# zfs set mountpoint=/var basefs/var
# zfs set mountpoint=/tmp basefs/tmp
# zfs set mountpoint=/usr/obj basefs/usr/obj
# zfs set mountpoint=/usr/ports basefs/usr/ports
# zfs set mountpoint=/usr/src basefs/usr/src
# zpool export basefs
# reboot
 
(9) Boot FreeBSD in single user mode.
Code:
# mount -w /
# zpool import -f basefs
# reboot

(10) Set root password
Code:
# passwd

(11) Set hostname
Code:
# echo hostname=\"HOSTNAME\" >> /etc/rc.conf

(12) Set timezone
Code:
# tzsetup

(13) Move termcap into /etc (instead of useless link on crash)
Code:
# rm /etc/termcap
# mv /usr/share/misc/termcap /etc
# ln -s /etc/termcap /usr/share/misc/termcap

(14) Setup networking and do any other necessary changes to /etc.

(15) Add latest security patches.
Code:
# freebsd-update fetch
# freebsd-update install

(16) Set / to be mounted read-only in /etc/fstab.
Code:
[color="Green"]+/dev/mirror/root /      ufs  ro    1     1   [/color]
[color="Red"]-/dev/mirror/root /      ufs  rw    1     1[/color]

(17) Reboot and enjoy.
Code:
# reboot
 
I never can understand why do you need create such a numbers of [zfs] partitions like /usr/ports, /usr/obj/, /usr/ports/distfiles and so on. It's excessive, doesn't neccesary for quick installation and obviusly can be done later.
This zfs partitioning way is in almost all howtos. Does anybody think the sense of it?
 
RusDyr said:
I never can understand why do you need create such a numbers of [zfs] partitions like /usr/ports, /usr/obj/, /usr/ports/distfiles and so on. It's excessive, doesn't neccesary for quick installation and obviusly can be done later.
This zfs partitioning way is in the almost each howtos. Does anybody think the sense of it?

With ZFS, these tasks are much easier indeed; using UFS-only setup you have to do it beforehand. But I think it's _the_ perfect place to do it during installation. You're not on time trial when installing new BSD, are you? It only takes couple of seconds to do it right.
 
RusDyr said:
What's the point? There is no difference in /usr/ports, /usr/obj and so on at least in current ZFS configuration.

There is a difference because if you want to set specific properties to those datasets, compression for example, you have to do it before they get populated.

/usr/src is a good candidate for compression but if you set this after the installation then you pretty much don't have any.

/usr/ports is another good candidate. But /usr/ports/distfiles is not since it usually contains compressed files.

Of course all those are suggestions and anybody can improvise.

Regards
 
RusDyr said:
What's the point? There is no difference in /usr/ports, /usr/obj and so on at least in current ZFS configuration.

For example, when you do a /usr snapshot. I don't care about /usr/ports/* , /usr/src/* at all.
You don't need /usr/ports for kernel debugging; /usr/obj can be shared to others too (when possible). So it does make sense to separate it.


Reasons for separate FS?

* backup/restore (do I need to back it up?) - restore time
* mount options (performance and/or security)
* implications when FS gets full or corrupted (can it be restored online ?)
 
Yeah, I thought about snapshots after I've wrote it.
Freeing space by compression or deduplication isn't valuable, especially on systems where are ZFS is (storage servers, isn't it?). ;)
 
RusDyr said:
Yeah, I thought about snapshots after I've wrote it.
Freeing space by compression or deduplication isn't valuable, especially on systems where are ZFS is (storage servers, isn't it?). ;)

Your original question was why all guides create so many ZFS datasets during installation, instead of doing this afterwards.

This has nothing to do with snapshots whatsoever. The only reason why you might want to create a dataset before installing the OS, is because you want to assign certain properties such as, compression, dedup, copies, etc.

I hope this clarifies your question.
 
I am trying this now. First time I tried the
Code:
# kldload geom_mbr.ko
It gave me a bunch of errors. however after I tried the second line, then went back to try it again, it says file aleady exists. So I hope it was loaded.

Now I am stuck at
Code:
# gpart backup $disk0 | gpart restore -F $disk1
because it says
Code:
gpart: geom 'ada1': Operation not permitted

dmesg says I have ada0 and ada1, both 1TB drives. It appears ada1 is not writable.
 
So I proceeded as if I had a single disk and ignored disk1. I had to omit 'mirror' and ${disk1} in the zpool command.

But now I tried exactly:

Code:
# cd /usr/freebsd-dist
# export DESTDIR=/tmp/NEWROOT
# for file in base.txz lib32.txz kernel.txz;
# do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done

and got:

Code:
cat: lib32.txz: no such file or directory.

I am using a freebsd FreeBSD 9.0 bootdisk (i386).
 
gkontos said:
The only reason why you might want to create a dataset before installing the OS, is because you want to assign certain properties such as, compression, dedup, copies, etc.

Now that's quite some harsh statement (especially "the only" part).
 
Back
Top