HOWTO: Install FreeBSD 9.0 RELEASE (Root on UFS + ZFS, RAID1)

vand777

Well-Known Member

Reaction score: 38
Messages: 292

I'll show how to install FreeBSD 9.0 RELEASE on the machine with 2 disks (RAID 1). I'll use the following logical layout:

Code:
UFS 1G   /           ro
ZFS *    /home       rw
ZFS *    /tmp        rw
UFS *    /usr        ro | softupdates (mounted r/w only for packages updates)
ZFS *    /usr/obj    rw 
ZFS *    /usr/ports  rw 
ZFS *    /usr/src    rw 
ZFS *    /var        rw
UFS 1G   /var/db/pkg ro | softupdates (mounted r/w only for packages updates)
 
OP
OP
V

vand777

Well-Known Member

Reaction score: 38
Messages: 292

(1) Boot from a FreeBSD9 installation DVD or memstick and choose "Live CD".

(2) Load the necessary kernel modules.
Code:
# kldload geom_mbr.ko
# kldload geom_mirror.ko
# kldload opensolaris.ko
# kldload zfs.ko
(3) Create the necessary partitions on the disks.
Code:
# sh
# disk0=da0
# disk1=da1
# gpart create -s gpt $disk0
# gpart add -b 34 -s 94 -t freebsd-boot $disk0
# gpart add -s 1G -t freebsd-ufs $disk0
# gpart add -s 4G -t freebsd-swap $disk0
# gpart add -s 1G -t freebsd-ufs $disk0
# gpart add -s 8G -t freebsd-ufs $disk0
# gpart add -t freebsd-zfs $disk0
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 $disk0
# gpart backup $disk0 | gpart restore -F $disk1
(4) Create RAID1 for UFS and swap filesystems.
Code:
# gmirror label  root  ${disk0}p2 ${disk1}p2
# gmirror label  swap  ${disk0}p3 ${disk1}p3
# gmirror label  pkg   ${disk0}p4 ${disk1}p4
# gmirror label  usr   ${disk0}p5 ${disk1}p5
 
OP
OP
V

vand777

Well-Known Member

Reaction score: 38
Messages: 292

(5) Create filesystems and mount them.
Code:
# newfs -m 1 /dev/mirror/root
# newfs -m 1 -U /dev/mirror/pkg
# newfs -m 1 -U /dev/mirror/usr
# mkdir /tmp/NEWROOT
# mount /dev/mirror/root /tmp/NEWROOT
# mkdir /tmp/NEWROOT/usr
# mount /dev/mirror/usr /tmp/NEWROOT/usr
# zpool create -m /tmp/NEWROOT/basefs basefs mirror ${disk0}p6 ${disk1}p6
# zfs create -o mountpoint=/tmp/NEWROOT/home basefs/home
# zfs create -o mountpoint=/tmp/NEWROOT/var basefs/var
# zfs create -o mountpoint=/tmp/NEWROOT/tmp basefs/tmp
# zfs create -o mountpoint=none basefs/usr
# zfs create -o mountpoint=/tmp/NEWROOT/usr/obj basefs/usr/obj
# zfs create -o mountpoint=/tmp/NEWROOT/usr/ports basefs/usr/ports
# zfs create -o mountpoint=/tmp/NEWROOT/usr/src basefs/usr/src
P.S. Feel free to play with ZFS filesystems options and create additional filesystems if you need/want.

(6) Install FreeBSD.
Code:
# cd /usr/freebsd-dist
# export DESTDIR=/tmp/NEWROOT
# for file in base.txz lib32.txz kernel.txz;
# do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
 
OP
OP
V

vand777

Well-Known Member

Reaction score: 38
Messages: 292

(7) Provide basic configuration.
Code:
# cat > /tmp/NEWROOT/etc/fstab << __EOF__             
#dev                #mount        #fs  #opts #dump #pass
/dev/mirror/root    /             ufs  rw    1     1     
/dev/mirror/swap    none          swap sw    0     0  
/dev/mirror/usr     /usr          ufs  rw    2     2  
basefs/var          /var          zfs  rw    0     0
/dev/mirror/pkg     /var/db/pkg   ufs  rw    2     2  

basefs              /basefs       zfs  rw    0     0
basefs/home         /home         zfs  rw    0     0
basefs/usr/obj      /usr/obj      zfs  rw    0     0
basefs/usr/ports    /usr/ports    zfs  rw    0     0
basefs/usr/src      /usr/src      zfs  rw    0     0
basefs/tmp          /tmp          zfs  rw    0     0
__EOF__   

# cat > /tmp/NEWROOT/boot/loader.conf << __EOF__      
zfs_load="YES"                                  
ahci_load="YES"                                 
geom_mirror_load="YES"                          
__EOF__  

# cat > /tmp/NEWROOT/etc/rc.conf << __EOF__
zfs_enable="YES"
__EOF__
(8) Unmount filesystems and reboot.
Code:
# cd /
# zfs umount -a
# umount /tmp/NEWROOT/usr
# umount /tmp/NEWROOT
# zfs set mountpoint=/home basefs/home
# zfs set mountpoint=/var basefs/var
# zfs set mountpoint=/tmp basefs/tmp
# zfs set mountpoint=/usr/obj basefs/usr/obj
# zfs set mountpoint=/usr/ports basefs/usr/ports
# zfs set mountpoint=/usr/src basefs/usr/src
# zpool export basefs
# reboot
 
OP
OP
V

vand777

Well-Known Member

Reaction score: 38
Messages: 292

(9) Boot FreeBSD in single user mode.
Code:
# mount -w /
# zpool import -f basefs
# reboot
(10) Set root password
Code:
# passwd
(11) Set hostname
Code:
# echo hostname=\"HOSTNAME\" >> /etc/rc.conf
(12) Set timezone
Code:
# tzsetup
(13) Move termcap into /etc (instead of useless link on crash)
Code:
# rm /etc/termcap
# mv /usr/share/misc/termcap /etc
# ln -s /etc/termcap /usr/share/misc/termcap
(14) Setup networking and do any other necessary changes to /etc.

(15) Add latest security patches.
Code:
# freebsd-update fetch
# freebsd-update install
(16) Set / to be mounted read-only in /etc/fstab.
Code:
[color="Green"]+/dev/mirror/root /      ufs  ro    1     1   [/color]
[color="Red"]-/dev/mirror/root /      ufs  rw    1     1[/color]
(17) Reboot and enjoy.
Code:
# reboot
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,264
Messages: 4,099

Just a quick note: if you have a separate /home, then you don't need the symlink.
 
OP
OP
V

vand777

Well-Known Member

Reaction score: 38
Messages: 292

phoenix said:
Just a quick note: if you have a separate /home, then you don't need the symlink.
Thank you. Will remove it from the guide in a second.
 

RusDyr

Member

Reaction score: 15
Messages: 90

I never can understand why do you need create such a numbers of [zfs] partitions like /usr/ports, /usr/obj/, /usr/ports/distfiles and so on. It's excessive, doesn't neccesary for quick installation and obviusly can be done later.
This zfs partitioning way is in almost all howtos. Does anybody think the sense of it?
 

_martin

Aspiring Daemon

Reaction score: 168
Messages: 814

RusDyr said:
I never can understand why do you need create such a numbers of [zfs] partitions like /usr/ports, /usr/obj/, /usr/ports/distfiles and so on. It's excessive, doesn't neccesary for quick installation and obviusly can be done later.
This zfs partitioning way is in the almost each howtos. Does anybody think the sense of it?
With ZFS, these tasks are much easier indeed; using UFS-only setup you have to do it beforehand. But I think it's _the_ perfect place to do it during installation. You're not on time trial when installing new BSD, are you? It only takes couple of seconds to do it right.
 

RusDyr

Member

Reaction score: 15
Messages: 90

What's the point? There is no difference in /usr/ports, /usr/obj and so on at least in current ZFS configuration.
 

gkontos

Daemon

Reaction score: 482
Messages: 2,155

RusDyr said:
What's the point? There is no difference in /usr/ports, /usr/obj and so on at least in current ZFS configuration.
There is a difference because if you want to set specific properties to those datasets, compression for example, you have to do it before they get populated.

/usr/src is a good candidate for compression but if you set this after the installation then you pretty much don't have any.

/usr/ports is another good candidate. But /usr/ports/distfiles is not since it usually contains compressed files.

Of course all those are suggestions and anybody can improvise.

Regards
 

_martin

Aspiring Daemon

Reaction score: 168
Messages: 814

RusDyr said:
What's the point? There is no difference in /usr/ports, /usr/obj and so on at least in current ZFS configuration.
For example, when you do a /usr snapshot. I don't care about /usr/ports/* , /usr/src/* at all.
You don't need /usr/ports for kernel debugging; /usr/obj can be shared to others too (when possible). So it does make sense to separate it.


Reasons for separate FS?

* backup/restore (do I need to back it up?) - restore time
* mount options (performance and/or security)
* implications when FS gets full or corrupted (can it be restored online ?)
 

bbzz

Aspiring Daemon

Reaction score: 133
Messages: 874

All above and zfs list show quickly how much space is taken in each fs :)
 

RusDyr

Member

Reaction score: 15
Messages: 90

Yeah, I thought about snapshots after I've wrote it.
Freeing space by compression or deduplication isn't valuable, especially on systems where are ZFS is (storage servers, isn't it?). ;)
 

gkontos

Daemon

Reaction score: 482
Messages: 2,155

RusDyr said:
Yeah, I thought about snapshots after I've wrote it.
Freeing space by compression or deduplication isn't valuable, especially on systems where are ZFS is (storage servers, isn't it?). ;)
Your original question was why all guides create so many ZFS datasets during installation, instead of doing this afterwards.

This has nothing to do with snapshots whatsoever. The only reason why you might want to create a dataset before installing the OS, is because you want to assign certain properties such as, compression, dedup, copies, etc.

I hope this clarifies your question.
 

beesatmsu

Well-Known Member

Reaction score: 2
Messages: 251

I am trying this now. First time I tried the
Code:
# kldload geom_mbr.ko
It gave me a bunch of errors. however after I tried the second line, then went back to try it again, it says file aleady exists. So I hope it was loaded.

Now I am stuck at
Code:
# gpart backup $disk0 | gpart restore -F $disk1
because it says
Code:
gpart: geom 'ada1': Operation not permitted
dmesg says I have ada0 and ada1, both 1TB drives. It appears ada1 is not writable.
 

beesatmsu

Well-Known Member

Reaction score: 2
Messages: 251

So I proceeded as if I had a single disk and ignored disk1. I had to omit 'mirror' and ${disk1} in the zpool command.

But now I tried exactly:

Code:
# cd /usr/freebsd-dist
# export DESTDIR=/tmp/NEWROOT
# for file in base.txz lib32.txz kernel.txz;
# do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
and got:

Code:
cat: lib32.txz: no such file or directory.
I am using a freebsd FreeBSD 9.0 bootdisk (i386).
 

beesatmsu

Well-Known Member

Reaction score: 2
Messages: 251

indeed, ls gives:
Code:
MANIFEST
doc.txz
kernel.txz
src.txz
base.txz
games.txz
ports.txz
but no lib32.txz.
 

_martin

Aspiring Daemon

Reaction score: 168
Messages: 814

gkontos said:
The only reason why you might want to create a dataset before installing the OS, is because you want to assign certain properties such as, compression, dedup, copies, etc.
Now that's quite some harsh statement (especially "the only" part).
 

beesatmsu

Well-Known Member

Reaction score: 2
Messages: 251

Oops, just found out my boot disk was 9.0-RC3, not 9.0-release as I thought. Will try again tomorrow.
 
Top