Replicating AUTO ZFS Partitioning Manually

Hey Guys,

Someone can explain in an step by step how the FreeBSD 10.1 Installer create the partitions on the option:

AUTO (ZFS) Guided Root-on-ZFS

For an single disk with encryption?
I was trying get same result manually on Installer using the shell without success.
Following this tutorial:

https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot

Got stuck on step 7

zpool create zroot /dev/gpt/disk0

With the error message:

Code:
invalid vdev specification
use '-f' to override the following errors:
/dev/gpt/disk0 is part of potentially active pool 'zroot'

If I do:

zpool create -f zroot /dev/gpt/disk0

Then I get this error:

Code:
cannot mount '/zroot/': failed to create mount point

Checking The default install in another virtual machine I have noticed the installer created 2 pools:

bootpool
zroot

So, for get same result I will need make 2 pools?
If yes, whats the ideal size of bootpool?
And there any special paramenter for use on gpart making the bootpool?
 
I recently scripted the part of the install where it relates to setting up the disk.

gkontos gave some very useful pointers

My script would be overly complex, for your needs as it was for 5 mag disk and 2 ssd, but the link he provided has a good options for single disk install or simple raids.

Please see

This Link

I didn't do encryption, but there may some documentation there to explain that, if not I could cobble something up for you as it would be just a couple more commands
 
Thanks for you answer Matthhew Dresden.

I did almost samething like your script, and got stuck on this error now on Archive Extraction Screen (file base.txz at 85%):

Code:
Error while extracting
base.txz: Can't restore time

Have tried FreeBSD 10.0 amd64 and FreeBSD 10.1 amd64 same error on both.

I have started by reading oracle documentation for get familiar with terminology and check for some hints:

https://docs.oracle.com/cd/E36784_01/pdf/E36835.pdf

After the read, I see the options for cache slice and log slice (before the doc dindt know about then), so I have choosed to grab both (I see those two in your script too Matthhew).
Then I start do the slices on this way:

Code:
gpart destroy -F da0
gpart create -s GPT da0
gpart show da0

gpart add -t freebsd-boot -b 40 -a 4K -s 512K -l gptboot0 da0
gpart add -t freebsd-zfs -b 263168 -a 4K -s 2G -l log0 da0
gpart add -t freebsd-zfs -b 4719616 -a 4K -s 8G -l cache0 da0
gpart add -t freebsd-swap -b 21758976 -a 4K -s 32G -l swap0 da0
gpart add -t freebsd-zfs -b 89129984 -a 4K -s 6G -l boot0 da0
gpart add -t freebsd-zfs -b 101975040 -a 4K -l zfs0 da0

gpart set -a bootme -i 5 da0

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

Phew necessary notes about this layout:

1 - The GAP (128MB) is an recommendation from MacOS Forums, they say this allow partition tools work better (dont gonna use those tools anyway)
2 - Some other sites say let an GAP unused allow the HD firmware use those for reallocate bad sectors (after almost 3 months of tests trying get FreeBSD as desktop I started get some bad blocks... And don't gonna buy an new HD till 2016, because I'm waiting Intel release they 100TB HDD and 10TB SSD), for my specific scenario i will isolate the bad blocks too letting then in unused space, so I will have this default layout GAP for future installs in new systems, and on my specific test notebook more GAPs for every bad sector.
3 - gptboot0 - Is the first slice like FreeBSD documentation says
4 - log0 - I have chosed this a second slice after read ZFS works like an database, and having some knowledge in databases, in my opinion the log would give a better performance remaining in the second inner slice from HDD - Oracle documentation says half of the physical RAM is an good value
5 - cache0 - Like log0, cache will be the next thing more used, so I grab this on third slice trying get better performance from inner sectors in HDD.
6 - swap0 - Like Linux, the mostly common advice is set it for the double of physical RAM, when I was using Ubuntu on this old notebook, with 32GB of SWAP I was able to compile Android ROMs (took almost 24 hours compiling but works), so I have chosed this value from this experience (Google says need have 16GB physical RAM)
7 - boot0 - The unencrypted slice, like all previous one, I'm using default labels from an regular install using the default options in installer
8 - zfs0 - Where gonna be the big pool handling with all files if i understood right.

PS.: I have FreeBSD as DNS server in an virtual machine and works pretty well, better than any Linux I have tried before, using less resources and more faster.

So after get the slices done, I have load the following modules:

Code:
kldload /boot/kernel/opensolaris.ko
kldload /boot/kernel/zfs.ko
kldload /boot/kernel/crypto.ko
kldload /boot/kernel/geom_eli.ko
kldload /boot/kernel/aesni.ko

And setup the following:

Code:
sysctl vfs.zfs.min_auto_ashift=12
sysctl vfs.zfs.max_auto_ashift=12

Note: FreeBSD 10.0 doesnt have: vfs.zfs.min_auto_ashift

So before start with the ZFS, I did the encryption:

Code:
dd if=/dev/random of=/tmp/bsdinstall_boot/encryption.key bs=128k count=1

geli init -b -K /tmp/bsdinstall_boot/encryption.key -s 4096 -l 256 /dev/gpt/log0
geli init -b -K /tmp/bsdinstall_boot/encryption.key -s 4096 -l 256 /dev/gpt/cache0
geli init -b -K /tmp/bsdinstall_boot/encryption.key -s 4096 -l 256 /dev/gpt/swap0
geli init -b -K /tmp/bsdinstall_boot/encryption.key -s 4096 -l 256 /dev/gpt/zfs0

geli attach -k /tmp/bsdinstall_boot/encryption.key /dev/gpt/log0
geli attach -k /tmp/bsdinstall_boot/encryption.key /dev/gpt/cache0
geli attach -k /tmp/bsdinstall_boot/encryption.key /dev/gpt/swap0
geli attach -k /tmp/bsdinstall_boot/encryption.key /dev/gpt/zfs0

Note: The default installer use /dev/da0p2 or /dev/da0p3 or /dev/da0p4 or /dev/da0p6. Looking in an virtual machine, there no devices like zroot swap on /dev/gpt. Till now I haven't found documentation about the correct way to setup this part, if using the devices inside /dev/gpt/ like all tutorials on internet, or like installer does.

Then, after get the slices encrypted (except the boot0 - this one if I understood right need be unencrypted for boot works), I have started setup the ZFS:

Code:
zpool create -O atime=on -O canmount=off -O checksum=fletcher4 -O compression=lz4 -O dedup=off -o failmode=panic -O normalization=formKC -O recordsize=128K -O reservation=1G bootpool gpt/boot0
zpool create -o altroot=/mnt -O atime=on -o autoexpand=on -O canmount=off -O checksum=fletcher4 -O compression=lz4 -O dedup=off -O failmode=panic -O normalization=formKC -O recordsize=128K -O reservation=10G zroot /dev/gpt/zfs0.eli

zpool list

zpool add zroot log gpt/log0.eli
zpool add zroot cache cache0.eli

zpool status zroot

zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ zroot/ROOT/default
zfs create -o mountpoint=/tmp -o setuid=off  zroot/tmp
chmod 1777 /mnt/tmp

zfs create -o mountpoint=/usr zroot/usr
zfs create zroot/usr/local
zfs create zroot/usr/jails
zfs create -o setuid=off zroot/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o exec=off -o setuid=off zroot/usr/src
zfs create zroot/usr/obj

zfs create -o mountpoint=/home -o setuid=off zroot/home

zfs create -o mountpoint=/var zroot/var
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o readonly=on -o setuid=off zroot/var/empty
zfs create -o exec=off -o setuid=off zroot/var/named
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o exec=on -o setuid=off zroot/var/tmp
chmod 1777 /mnt/var/tmp

zpool set bootfs=zroot/ROOT/default zroot

cat << EOF > /tmp/bsdinstall_etc/fstab
# Device     Mountpoint   FStype   Options     Dump   Pass#
/dev/gpt/swap0.eli     none     swap   sw     0   0

The I do EXIT for the installer continue, and the error occours on Archive Extraction Screen (file base.txz at 85%):

Code:
Error while extracting
base.txz: Can't restore time

Looking on google seems this is an bug from 10.0 "fixed" on 10.1...

Some sites I have searched for:

https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE
http://blather.michaelwlucas.com/archives/1982
https://www.keltia.net/howtos/mfsbsd-zfs91/
https://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061388.html
https://pthree.org/2012/12/18/zfs-administration-part-xi-compression-and-deduplication/

PS.: I haven't read the beadm documentation yet, but this seems be amazing for test the releases and be able to run back for previous versions, so, my goal will be have this one ready for use too, any advice point me about errors I would have did I will be grateful.
(For example, reading PC-BSD forums I have see the 11.0 release is using the new nvidia drivers, my old notebook doesnt work well with those drivers... On Ubuntu I was stucked on 12.0 - using nv drivers and nvidia binary - because the drivers, on 14.0 removing the "new" drivers and installing old ones give poor performance too..)
 
Just in case someone else gets the same error of:

Code:
Error while extracting
base.txz: Can't restore time

This means it can't extract files in folder /var/empty.

Because it was set up as read-only with:

zfs create -o exec=off -o readonly=on -o setuid=off zroot/var/empty

And finally was able to install following this guide:

http://www.schmidp.com/2014/01/07/zfs-full-disk-encryption-with-freebsd-10-part-2/

Going back to process of trying to install Gnome from source now, thanks for the help.

I'm curious now if after the boot should let as read only with:

zfs set readonly=on zroot/var/empty

Thinking about new base updates (If I'm going to remember this parameter to remove read-only again).
 
Back
Top