IN-DETAIL INSTRUCTION SET:
Locate and identify your drives (example):
Code:
[CMD="#"]camcontrol devlist[/CMD]
<ATA SOMETHING 0001> at scbus0 target 0 lun 0 (ada0,pass0) (Disk B?)
<ATA SOMETHING 0001> at scbus0 target 1 lun 0 (ada1,pass1) (Disk C?)
<ATA SOMETHING 0001> at scbus0 target 2 lun 0 (ada2,pass2) (Disk D?)
<ATA SOMETHING 0001> at scbus0 target 3 lun 0 (ada3,pass3) (Disk E?)
<USB SOMETHING 1.0> at scbus0 target 4 lun 0 (pass4,da0) (USB A?)
<USB SOMETHING 1.0> at scbus0 target 5 lun 0 (pass5,da1) (USB B?)
If you want to know more about one of them, you can use e.g.:
# diskinfo -v da0
Load up gmirror, identify the partitions and slices, check the filesystem/s, mount and then split the mirror to free up a drive to use for your new data-pool:
# gmirror load
Code:
[CMD="#"]ls -1 /dev/mirror[/CMD]
gm0
gm0s1
gm0s1a
# fsck -t ufs /dev/mirror/gm0s1a
Repeat for each slice, if more than just the "a".
# gmirror remove gm0 /dev/ada1
# gpart delete -i 1 ada1
# gpart destroy ada1
Now to check if there´s any partitioning on D or E (the sizes and sectors are different from yours, it´s just an example):
Code:
[CMD="#"]gpart show ada2[/CMD]
=> 34 8388541 ada2 BSD (4.0G)
34 8388541 1 freebsd-ufs (4G)
[CMD="#"]gpart show ada3[/CMD]
=> 34 8388541 ada3 BSD (4.0G)
34 8388541 1 freebsd-ufs (4G)
And if any, clean them:
# gpart delete -i 1 ada2
# gpart delete -i 1 ada3
# gpart destroy ada2
# gpart destroy ada3
Create new partitioning scheme and set partition start- and stop aligned after 4k:
# gpart create -s gpt ada1
# gpart create -s gpt ada2
# gpart create -s gpt ada3
# gpart add -t freebsd-zfs -l diskC -b 2048 -a 4k ada1
# gpart add -t freebsd-zfs -l diskD -b 2048 -a 4k ada2
# gpart add -t freebsd-zfs -l diskE -b 2048 -a 4k ada3
This is a tricky part. For retarted reasons, you need to create "fake" drives to create your pool with. You shouldn´t have to do this really, but it´s the only workaround I´ve found for the
gnop-trick to work and the labels to show up. The recipe for
dd is to seek forwards to your total size of your drives, minus one MB, for the partition start- and stop boundary. Use
diskinfo to see the size of one of your drives in bytes:
Code:
[CMD="#"]diskinfo -v ada0[/CMD]
512 # sectorsize
1000199467008 # mediasize in bytes (1T)
1953514584 # mediasize in sectors
... ( I didn´t have any 1TB drives available, so these numbers are based on a 2TB drive divided by 2 )
[CMD="#"]echo "1000199467008 / 1024000 - 1" | bc[/CMD]
976756
# dd if=/dev/zero of=/tmp/tmpdsk2 bs=1m seek=976756 count=1
# dd if=/dev/zero of=/tmp/tmpdsk3 bs=1m seek=976756 count=1
# dd if=/dev/zero of=/tmp/tmpdsk4 bs=1m seek=976756 count=1
# dd if=/dev/zero of=/tmp/tmpdsk5 bs=1m seek=976756 count=1
# mdconfig -a -t vnode -f /tmp/tmpdsk2 md2
# mdconfig -a -t vnode -f /tmp/tmpdsk3 md3
# mdconfig -a -t vnode -f /tmp/tmpdsk4 md4
# mdconfig -a -t vnode -f /tmp/tmpdsk5 md5
Do the files need to be called the same as the md´s? No, but it´s alot easier to remember what was what.
# gnop create -S 4096 md2
# zpool create -O mountpoint=none -o autoexpand=on -o cachefile=/var/tmp/zpool.cache pool2 raidz md2.nop md{3,4,5}
# zpool export pool2
# gnop destroy md2.nop
# zpool import -o cachefile=/var/tmp/zpool.cache pool2
So the data-pool is now imported again with the "fake" drives. By using the
gnop-trick, we tell ZFS that the smallest write it can send is 4k. Now we begin to replace the fake ones with the real, minus drive B which we will have to deal with after we´ve copied the data out from it.
# zpool offline pool2 md2
# mdconfig -d -u 2
# rm /tmp/tmpdsk2
# zpool offline pool2 md3
# mdconfig -d -u 3
# rm /tmp/tmpdsk3
# zpool replace pool2 md3 gpt/diskC
# zpool offline pool2 md4
# mdconfig -d -u 4
# rm /tmp/tmpdsk4
# zpool replace pool2 md4 gpt/diskD
# zpool offline pool2 md5
# mdconfig -d -u 5
# rm /tmp/tmpdsk5
# zpool replace pool2 md5 gpt/diskE
Using the same method to create the boot-pool:
# gpart create -s gpt da0
# gpart create -s gpt da1
# gpart add -t freebsd-zfs -l usbA -b 2048 -a 4k da0
# gpart add -t freebsd-zfs -l usbB -b 2048 -a 4k da1
# dd if=/dev/zero of=/tmp/tmpdsk2 bs=1m seek=8191 count=1
# mdconfig -a -t vnode -f /tmp/tmpdsk2 md2
# gnop create -S 4096 md2
# zpool create -O mountpoint=none -o cachefile=/var/tmp/zpool.cache pool1 md2.nop
# zpool export pool1
# gnop destroy md2.nop
# zpool import -o cachefile=/var/tmp/zpool.cache pool1
# zpool attach pool1 md2 gpt/usbB
# zpool offline pool1 md2
# mdconfig -d -u 2
# rm /tmp/tmpdsk2
# zpool replace pool1 md2 gpt/usbA
# zfs create -o mountpoint=legacy -o compress=on pool1/root
# zfs create -o mountpoint=legacy -o compress=on pool2/root
# zfs create pool2/root/usr
# zfs create pool2/root/usr
# zfs create pool2/root/usr/local
# zfs create pool2/root/usr/home
# zfs create pool2/root/var
# zfs create -o compress=on -s -V 512m pool2/swap
Swap is historically decided on "your amount of RAM"x2
# zpool set bootfs=pool1/root pool1
# mount -t zfs pool1/root /mnt
# mkdir /mnt/tmp
# mkdir /mnt/usr
# mkdir /mnt/var
# mount -t zfs pool2/root/usr /mnt/usr
# mount -t zfs pool2/root/var /mnt/var
# mkdir /mnt/usr/home
# mkdir /mnt/usr/local
# mount -t zfs pool2/root/usr/home /mnt/usr/home
# mount -t zfs pool2/root/usr/local /mnt/usr/local
Edit the configuration files needed for the system to boot up and mount everything in place:
Code:
[CMD="#"]ee /tmp/bsdinstall_etc/fstab[/CMD]
/dev/zvol/pool2/swap none swap sw 0 0
pool1/root / zfs rw 0 0
tmpfs /tmp tmpfs rw 0 0
pool2/root/usr /usr zfs rw 0 0
pool2/root/usr/home /usr/home zfs rw 0 0
pool2/root/usr/local /usr/local zfs rw 0 0
pool2/root/var /var zfs rw 0 0
[CMD="#"]ee /mnt/boot/loader.conf[/CMD]
autoboot_delay="5"
zfs_load="YES"
vfs.root.mountfrom="zfs:pool1/root"
Now we start copying everything out from the old BC-mirror:
# mkdir -p /mnt/usr/home/BC-mirror/gm0s1a
# mount -t ufs /dev/mirror/gm0s1a /media
# cd /media
# find -xs . | cpio -pv /mnt/usr/home/BC-mirror/gm0s1a
# cd /
# umount /media
Repeat for each slice, if more than just the "a".
Crash the old mirror, clean partitioning, create new partitioning and complete the data-pool:
# gmirror stop gm0
# gmirror unload gm0
# gpart delete -i 1 ada0
# gpart destroy ada0
# gpart create -s gpt ada0
# gpart add -t freebsd-zfs -l diskB -b 2048 -a 4k ada0
# zpool replace pool2 md2 gpt/diskB
Resilvering this disk can take a while depending on much you copied in from the old BC mirror. Watch the progress with:
# zpool status pool2
# zpool iostat pool2 1
When done, wrap it up with:
# mkdir -p /mnt/boot/zfs
# cp /var/tmp/zpool.cache /mnt/boot/zfs/
# exit
I have tried to describe the process as detailed as possible, and in doing so there´s also a greater risk of typos. I have re-read this a couple of times to minimize this risk and I have also gone through this exact process in a VMWare virtual machine, except with a smaller disk size (4GB), and it worked from start to finish. I have taken as many things as I can into consideration, still there might be things that differs when you attempt this; device names e.g. Either you rearrange how the drives are connected in you chassis to match the instruction set, or you correct the commands as appropriate.
May the Schwartz be with you, always.
/Sebulon
