graudeejs
August 12th, 2009, 17:29
I've been using zfs for some time now on my desktop pc. And finally I decided to share some tips.
Before we start, you need to know that I use geli to encrypt my HDD's. I boot from usb flash. And what I tell may depend on that.
I don't intend to explain how to start using ZFS, I want to tell you how to use it and save some problems later.
Let's imagine we have fresh zpool named mzfs.
1) set zfs options for your pool that you think would be default in most cases
For example, I set:
zfs set copies=2 mzfs
zfs set compression=gzip mzfs
New zfs file systems will use zfs settings from parent file system (in this case zpool root), you can override them for other file systems
In addition I do
zfs set mountpoint=none mzfs
2) Don't install FreeBSD to root of pool.
create mzfs/root and install FreeBSD there.
[I will explain later]
3) group jails
To group my jails I create filesyste
zfs create mzfs/jails
zfs set mountpoint=/jails mzfs/jails
set other options as you need.
Child file systems will have parent fs settings, and will be auto mounted
4) create snapshot for 1 jail use for all of them
Now let's imagine you have build your system from sources and want to set up jails.
First thing you need to do is create 1 jails, that will be used for all other jails as starting point.
zfs create mzfs/jails/testbox
cd /usr/src
make buildworld
make installworld DESTDIR=/jails/testbox
cd etc
make distribution DESTDIR=/jails/testbox
cd /jails/testbox/etc
touch fstab
after this edit all files in /jails/testbox/etc that you nedd (rc.conf, resolv.conf)
once you have finished editing files, this is the point every jail will have common.
Create zfs snapshoot
zfs snapshot mzfs/jails/testbox@fresh
now, when you need fresh jail for you server simply run
zfs clone mzfs/jails/testbox@fresh mzfs/jails/server
this jail will be auto mounted
I also keep /jails/fstab.JAIL_NAME files (replace JAIL_NAME with jail name)
here's sample:
/usr/ports /jails/testbox/usr/ports nullfs noatime,rw 0 0
/usr/src /jails/testbox/usr/src nullfs noatime,ro 0 0
here's sample from my rc.conf
jail_testbox_hostname="testbox"
ifconfig_rl0_alias0="inet 192.168.128.99 netmask 0xffffff00"
jail_testbox_ip="192.168.128.99"
jail_testbox_rootdir="/jails/testbox"
jail_testbox_exec="/bin/sh /etc/rc"
jail_testbox_devfs_enable="YES"
jail_testbox_fstab="/jails/fstab.testbox"
jail_testbox_mount_enable="YES"
when I create new jail I simply copy this block and edit it.
and copy /jails/fstab.testbox to /jails/fstab.server
add new jail to jail list and start it.
creating new jails with zfs is very easy, and best of all it takes zero space.
5) Rebuild system from jail
Now If you want to have fresh FreeBSD, you can build it in jail
simply do what's written in tip 4
after you installed all apps from ports (or whatever you did)
stop jail and then edit loader.conf
change
vfs.root.mountfrom="zfs:mzfs/root"
to
vfs.root.mountfrom="zfs:mzfs/root2"
don't forget to edit your /jails/newfreebsd/etc/rc.conf and /jails/newfreebsd/etc/fstab, or copy them form your old freebsd
cp /etc/rc.conf /jails/newfreebsd/etc/rc.conf
cp /etc/fstab /jails/newfreebsd/etc/fstab
do same for other files as well, that you have modified.
and finally run
zfs rename zfs/jails/newfreebsd zfs/root2
next time you reboot you will boot in your newly created FreeBSD.
This way you will avoid downtime, and won't have to destroy zpool (pretty much no matter what)
after reboot you can destroy old system
zfs destroy a/root
6) you may not agree with this
On zfs I prefer to not separate /usr /usr/local etc from root. Basically I keep everything I need to run FreeBSD as Desktop under 1 Filsystem.
I separate /tmp /usr/src /usr/ports /usr/obj /var/db/portsnap becaue i use different zfs options on them (copies=1 for example)
NOTE: don't forget to install kernel on usb flash ;) [or the media you're using to boot from]
Fallowing these tips I installed fresh FreeBSD with KDE4.3, just to test KDE, and later switched back to my good old fvwm.
I attached tar with my config files, to help you understand what I was saying here.
Note, that my fstab is empty. It really is empty on my pc.
P.S.
If you have questions, post here, I'll answer
P.S.S.
Now think what would you do if you had your FreeBSD root on zpool root?
UPDATE:
7) compression=off
Compression sounds, good, but in reality it isn't all that good.
1.) you can't boot from compressed pool
2.) when i was copying data to compressed pool on flash I had huge lags (well compression was high, but still, lags ware HUGE, really HUGE). When I was copying same data, to same flash, but with compression turned off, lags were gone.
From now on, i will use compression only on very few parts of systems (ports tree, logs....)
UPDATE:
Remember that if you need to work with zfs in fixit mode (from fixit cd or dvd), you need to
# mkdir /boot/zfs
# ln -s /dist/boot/kernel /boot/kernel
# ln -s /dist/lib /lib
# kldload zfs
Also remember, that you probably won't be able to boot from exported zfs!
UPDATE:
If you want to make periodic zfs snapshots you may want to try sysutils/zfsnap
UPDATE:
in /etc/contab
@reboot root /usr/local/sbin/zfSnap -a 1w -r a/home a/root a/local
Before we start, you need to know that I use geli to encrypt my HDD's. I boot from usb flash. And what I tell may depend on that.
I don't intend to explain how to start using ZFS, I want to tell you how to use it and save some problems later.
Let's imagine we have fresh zpool named mzfs.
1) set zfs options for your pool that you think would be default in most cases
For example, I set:
zfs set copies=2 mzfs
zfs set compression=gzip mzfs
New zfs file systems will use zfs settings from parent file system (in this case zpool root), you can override them for other file systems
In addition I do
zfs set mountpoint=none mzfs
2) Don't install FreeBSD to root of pool.
create mzfs/root and install FreeBSD there.
[I will explain later]
3) group jails
To group my jails I create filesyste
zfs create mzfs/jails
zfs set mountpoint=/jails mzfs/jails
set other options as you need.
Child file systems will have parent fs settings, and will be auto mounted
4) create snapshot for 1 jail use for all of them
Now let's imagine you have build your system from sources and want to set up jails.
First thing you need to do is create 1 jails, that will be used for all other jails as starting point.
zfs create mzfs/jails/testbox
cd /usr/src
make buildworld
make installworld DESTDIR=/jails/testbox
cd etc
make distribution DESTDIR=/jails/testbox
cd /jails/testbox/etc
touch fstab
after this edit all files in /jails/testbox/etc that you nedd (rc.conf, resolv.conf)
once you have finished editing files, this is the point every jail will have common.
Create zfs snapshoot
zfs snapshot mzfs/jails/testbox@fresh
now, when you need fresh jail for you server simply run
zfs clone mzfs/jails/testbox@fresh mzfs/jails/server
this jail will be auto mounted
I also keep /jails/fstab.JAIL_NAME files (replace JAIL_NAME with jail name)
here's sample:
/usr/ports /jails/testbox/usr/ports nullfs noatime,rw 0 0
/usr/src /jails/testbox/usr/src nullfs noatime,ro 0 0
here's sample from my rc.conf
jail_testbox_hostname="testbox"
ifconfig_rl0_alias0="inet 192.168.128.99 netmask 0xffffff00"
jail_testbox_ip="192.168.128.99"
jail_testbox_rootdir="/jails/testbox"
jail_testbox_exec="/bin/sh /etc/rc"
jail_testbox_devfs_enable="YES"
jail_testbox_fstab="/jails/fstab.testbox"
jail_testbox_mount_enable="YES"
when I create new jail I simply copy this block and edit it.
and copy /jails/fstab.testbox to /jails/fstab.server
add new jail to jail list and start it.
creating new jails with zfs is very easy, and best of all it takes zero space.
5) Rebuild system from jail
Now If you want to have fresh FreeBSD, you can build it in jail
simply do what's written in tip 4
after you installed all apps from ports (or whatever you did)
stop jail and then edit loader.conf
change
vfs.root.mountfrom="zfs:mzfs/root"
to
vfs.root.mountfrom="zfs:mzfs/root2"
don't forget to edit your /jails/newfreebsd/etc/rc.conf and /jails/newfreebsd/etc/fstab, or copy them form your old freebsd
cp /etc/rc.conf /jails/newfreebsd/etc/rc.conf
cp /etc/fstab /jails/newfreebsd/etc/fstab
do same for other files as well, that you have modified.
and finally run
zfs rename zfs/jails/newfreebsd zfs/root2
next time you reboot you will boot in your newly created FreeBSD.
This way you will avoid downtime, and won't have to destroy zpool (pretty much no matter what)
after reboot you can destroy old system
zfs destroy a/root
6) you may not agree with this
On zfs I prefer to not separate /usr /usr/local etc from root. Basically I keep everything I need to run FreeBSD as Desktop under 1 Filsystem.
I separate /tmp /usr/src /usr/ports /usr/obj /var/db/portsnap becaue i use different zfs options on them (copies=1 for example)
NOTE: don't forget to install kernel on usb flash ;) [or the media you're using to boot from]
Fallowing these tips I installed fresh FreeBSD with KDE4.3, just to test KDE, and later switched back to my good old fvwm.
I attached tar with my config files, to help you understand what I was saying here.
Note, that my fstab is empty. It really is empty on my pc.
P.S.
If you have questions, post here, I'll answer
P.S.S.
Now think what would you do if you had your FreeBSD root on zpool root?
UPDATE:
7) compression=off
Compression sounds, good, but in reality it isn't all that good.
1.) you can't boot from compressed pool
2.) when i was copying data to compressed pool on flash I had huge lags (well compression was high, but still, lags ware HUGE, really HUGE). When I was copying same data, to same flash, but with compression turned off, lags were gone.
From now on, i will use compression only on very few parts of systems (ports tree, logs....)
UPDATE:
Remember that if you need to work with zfs in fixit mode (from fixit cd or dvd), you need to
# mkdir /boot/zfs
# ln -s /dist/boot/kernel /boot/kernel
# ln -s /dist/lib /lib
# kldload zfs
Also remember, that you probably won't be able to boot from exported zfs!
UPDATE:
If you want to make periodic zfs snapshots you may want to try sysutils/zfsnap
UPDATE:
in /etc/contab
@reboot root /usr/local/sbin/zfSnap -a 1w -r a/home a/root a/local