zfs_setup.sh

D

da1

Hi guys,

As a hobby, I'm playing with a script to automate the installation of a RooTonZFS FreeBSD. So far, I have gotten to make the mirror and the single/stripe setup to work like I want it. Since I'm not that good in scripting (but learning while writing code), I reached a bump in the road and need assistance (after 2 days of google-ing).

The problem:
when selecting raidz(1,2 or 3) and after inputing the hdd's to be used, the script asks the user to input the label for the disks. This is where the tricky part comes in as I have no idea how to write the code so that the script will know hot to match the first hdd with the first label (second hdd with the second label, etc) and then assign a variable to each of then and build the gpart cmd (gpart add -t freebsd-zfs -l <label> <hdd>). For the mirror part, I have done it manually because there will only be 2 disks and 2 labels no matter the case but for raidz(1,2,3) the number of disks can vary so I need some dynamic code here.

The script:
Code: 
#!/bin/sh

# DONE:
# - implement hdd autodetection and autocompletion (DONE)
#
# TO-DO:
# - implement mbr as an option to GPT (50% done)
# - implement swap-gpt and swap-zfs
# - fix raidz1,2,3 boot problem 
# - implement "tzsetup" and (root's) "passwd" to run via chroot cmd
# - ncurses? menu to allow setting dif. options (montpoints with options 
# (ex: compresion, etc), etc)
# 



#########################
# PRE-DEFINED FUNCTIONS #
#########################
release="uname -r"
partition_table_scheme="GPT"
swap_size="4G"
swap_label="swap"
zroot_label="zroot"

# Get a list of hdd's
get_disks=`dmesg | awk '{print \$1}' | egrep 'ad[0-9]|da[0-9]' | grep -v ":" | uniq`
for i in `echo $get_disks`
do
   hdd_list="$hdd_list $i"
done


link() {
if [ ! -h /boot/kernel ];then
ln -s /dist/boot/kernel /boot/kernel
fi
 
if [ ! -h /lib ];then
ln -s /dist/lib /lib
fi
}

kldload_modules(){
kldstat | grep opensolaris >> /dev/null
if [ $? != "0" ];then
kldload /mnt2/boot/kernel/opensolaris.ko
fi

kldstat | grep zfs >> /dev/null
if [ $? != "0" ];then
kldload /mnt2/boot/kernel/zfs.ko
fi
}

gpt_hdd() {
echo "Pls manually destroy any potential GPT partitions. Continuing after 5 seconds."
sleep 5

for i in $hdd
do
gpart create -s gpt $i
gpart add -s 128 -t freebsd-boot $i
gpart add -s $swap_size -t freebsd-swap -l $swap_label $i
gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 $i
done

case $zfs_setup in
   single|stripe)
	;;
   mirror) gpart add -t freebsd-zfs -l $label_hdd1 $hdd1
   	   gpart add -t freebsd-zfs -l $label_hdd2 $hdd2
	;;
   raidz1)
	;;
   raidz2)
	;;
   raidz3)
	;;
esac
}

#mbr_hdd() {
#}

create_zpool() {
case $zfs_setup in
        single|stripe) zpool create -f $zroot_label /dev/gpt/$hdd_label
		zpool set bootfs=$zroot_label $zroot_label
        ;;
        mirror) zpool create -f $zroot_label $zfs_setup /dev/gpt/$label_hdd1 /dev/gpt/$label_hdd2
		zpool set bootfs=$zroot_label $zroot_label
        ;;
        raidz1) zpool create -f $zroot_label $zfs_setup $raidz1_hdd_label
	        zpool set bootfs=$zroot_label $zroot_label
        ;;
        raidz2) zpool create -f $zroot_label $zfs_setup $raidz2_hdd_label
		zpool set bootfs=$zroot_label $zroot_label
	;;
        raidz3) zpool create -f $zroot_label $zfs_setup $raidz3_hdd_label
		zpool set bootfs=$zroot_label $zroot_label
	;;
esac
}

create_zroot_fs() {
set checksum=fletcher4 $zroot_label
zfs create -o compression=on -o exec=on -o setuid=off $zroot_label/tmp
chmod 1777 /$zroot_label/tmp
zfs create $zroot_label/usr
zfs create $zroot_label/usr/home
cd /$zroot_label; ln -s /usr/home home
zfs create -o compression=lzjb -o setuid=off $zroot_label/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off $zroot_label/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off $zroot_label/usr/ports/packages
zfs create $zroot_label/var
zfs create -o compression=lzjb -o exec=off -o setuid=off $zroot_label/var/crash
zfs create -o exec=off -o setuid=off $zroot_label/var/db
zfs create -o compression=lzjb -o exec=on -o setuid=off $zroot_label/var/db/pkg
zfs create -o exec=off -o setuid=off $zroot_label/var/empty
zfs create -o compression=lzjb -o exec=off -o setuid=off $zroot_label/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off $zroot_label/var/mail
zfs create -o exec=off -o setuid=off $zroot_label/var/run
zfs create -o compression=lzjb -o exec=on -o setuid=off $zroot_label/var/tmp
chmod 1777 /$zroot_label/var/tmp
}

extract_cd() {
cd /dist/`$release`/
export DESTDIR=/$zroot_label
for dir in base manpages ; do (cd $dir ; ./install.sh) ; done
cd src ; ./install.sh all
cd ../kernels ; ./install.sh generic
cd /$zroot_label/boot ; cp -Rlp GENERIC/* /$zroot_label/boot/kernel/
}

populate_loader.conf() {
echo "\
ahci_load=\"YES\"
atapicam_load=\"YES\"

#ZFS kernel modules
zfs_load=\"YES\"
vfs.root.mountfrom=\"zfs:$zroot_label\"

# ZFS tunnables
## for AHCI
vfs.zfs.vdev.min_pending=4              #default=4
vfs.zfs.vdev.max_pending=8              #default = 35
## NO AHCI
#vfs.zfs.vdev.min_pending=4             #default=4
#vfs.zfs.vdev.max_pending=8             #default = 35

# Increase vm.kmem_size to allow for ZFS ARC to utilise more memory.
# vm.kmem_size=\"2048M\"                  # Not needed in 8.2
# vm.kmem_size_max=\"2048M\"              # Not needed in 8.2
vfs.zfs.arc_max=\"2048M\"

# Disable ZFS prefetching (we will not disable it because we have 6GB of RAM)
# http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html
# vfs.zfs.prefetch_disable=\"0\"
                
# Decrease ZFS txg timeout value from 30 (default) to 5 seconds.  This
# should increase throughput and decrease the \"bursty\" stalls that
# happen during immense I/O with ZFS.
# http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html
# http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html
vfs.zfs.txg.timeout=\"5\"

# Target seconds to sync a txg
vfs.zfs.txg.synctime=\"1\"
" > /$zroot_label/boot/loader.conf
}

populate_rc.conf() {
echo "\
zfs_enable=\"YES\"
sshd_enable=\"YES\"
" > /$zroot_label/etc/rc.conf
}

export_import_zpool() {
mkdir /boot/zfs
cd /boot/zfs
zpool export $zroot_label && zpool import $zroot_label
cp /boot/zfs/zpool.cache /$zroot_label/boot/zfs/zpool.cache
echo "/dev/gpt/$swap_label none swap sw 0 0" > /$zroot_label/etc/fstab
export LD_LIBRARY_PATH=/mnt2/lib
}

set_zpool_mountpoints() {
zfs unmount -a
zfs set mountpoint=legacy $zroot_label
zfs set mountpoint=/tmp $zroot_label/tmp
zfs set mountpoint=/usr $zroot_label/usr
zfs set mountpoint=/var $zroot_label/var
}


clear
echo 'ZFS data parity scheme [single or stripe, mirror, raidz1, raidz2, raidz3] ?'
read zfs_setup
case $zfs_setup in
        single|stripe) 
		while [ -z "$hdd" ]
		do
		   echo -n "Hdd [$hdd_list] "
		   read hdd
		done
		echo -n "hdd label [$hdd] "
                read hdd_label
                   if [ -z "$hdd_label" ];then
                      hdd_label=$hdd
                   fi
        ;;
        mirror) 
                while [ -z "$hdd" ]
                do
                   echo -n "Hdd [$hdd_list] "
                   read hdd
                done
                hdd1="`echo $hdd | awk '{print \$1}'`"
                hdd2="`echo $hdd | awk '{print \$2}'`"

		echo -n "hdd label [$hdd] "
		read hdd_label
		   if [ -z "$hdd_label" ];then
                      label_hdd1=$hdd1
                      label_hdd2=$hdd2
		   else
		      label_hdd1="`echo $hdd_label | awk '{print \$1}'`"
		      label_hdd2="`echo $hdd_label | awk '{print \$2}'`"
		   fi
        ;;
        raidz1) 
                while [ -z "$hdd" ]
                do
                   echo -n "Hdd [$hdd_list] "
                   read hdd
                done
	       echo -n "hdd label [$hdd] "
		read hdd_label
                   if [ -z "$hdd_label" ];then
                      hdd_label=$hdd
                   fi
	       for i in $hdd_label;
		do
		   raidz1_hdd_label="$raidz1_hdd_label /dev/gpt/"$i""
		done
        ;;
        raidz2) 
                while [ -z "$hdd" ]
                do
                   echo -n "Hdd [$hdd_list] "
                   read hdd
                done
               echo -n "hdd label [$hdd] "
                read hdd_label
                   if [ -z "$hdd_label" ];then
                      hdd_label=$hdd
                   fi
               for i in $hdd_label;
                do
                   raidz2_hdd_label="$raidz2_hdd_label /dev/gpt/"$i""
                done
        ;;
        raidz3) 
                while [ -z "$hdd" ]
                do
                   echo -n "Hdd [$hdd_list] "
                   read hdd
                done
               echo -n "hdd label [$hdd] "
                read hdd_label
                   if [ -z "$hdd_label" ];then
                      hdd_label=$hdd
                   fi
               for i in $hdd_label;
                do
                   raidz3_hdd_label="$raidz3_hdd_label /dev/gpt/"$i""
                done
        ;;
        *) echo "Wrong option. Exiting ..."
           exit 1
        ;;
esac

echo -n "Partition table scheme: MBR or GPT [GPT] ? "
read newpartition_table_scheme
if [ ! -z $newpartition_table_scheme ];then
   partition_table_scheme=$newpartition_table_scheme
fi

echo -n "Swap size ["$swap_size"] "
read newswap_size
if [ ! -z $newswap_size ];then
   swap_size=$newswap_size
fi

echo -n "Swap label [swap] "
read newswap_label
if [ ! -z $newswap_label ];then
   swap_label=$newswap_label
fi

echo -n "zroot label [zroot] "
read newzroot_label
if [ ! -z $newzroot_label ];then
   zroot_label=$newzroot_label
fi
 
part2

Code: 
# Last check
echo ""
echo "\
You have selected:

Partition table scheme:  $partition_table_scheme
ZFS partition scheme:    $zfs_setup
HDD's:      $hdd"

case $zfs_setup in
   mirror) echo "HDD labels:: $label_hdd1 $label_hdd2"
	;;
esac

echo "\
Swap size:  $swap_size
Swap label: $swap_label
ZFS root partition name: $zroot_label"

echo -n "Continue [yes] ?"
read answer

case "$answer" in
   no) echo "Exiting ..."
	exit 1
	;;
esac


# Execute functions

link
kldload_modules

case $partition_table_scheme in
   gpt|GPT) gpt_hdd
	;;
   mbr|MBR) mbr_hdd
	;;
esac

create_zpool
create_zroot_fs
extract_cd
zfs set readonly=on $zroot_label/var/empty
populate_loader.conf
populate_rc.conf
export_import_zpool
set_zpool_mountpoints



echo "\
/boot/loader.conf, /etc/fstab and /etc/rc.conf were populated \
with some data. Make sure to check and adjust them as needed. \
Also make sure to set root's password and the time zone (tzsetup) \
after you boot up the new system."


#########################################################
############### ZFS in single user mode #################
###################  or in safe mode  ###################
#########################################################
#ln -s /dist/boot/kernel /boot/kernel
#ln -s /dist/lib /lib
#kldload /mnt2/boot/kernel/opensolaris.ko
#kldload /mnt2/boot/kernel/zfs.ko
#zpool import -f zroot
#zfs set mountpoint=/zroot zroot

One requirement, so far, is for the script to use sh. If you have suggestions how to improve my script, they are of course, welcomed.

Thx in advance.
 
Using:
Code: 
set -- $hdd
for i in $hdd_label
do
  gpart add -t freebsd-zfs -l $i $1
shift
done
 
Hi neighbor da1!

I haven't worked through your script entirely (just cherry picked a bit) but I suggest you to use something like $(sysctl -n kern.disks) to get the list of probed disk devices (instead of grep'ing through dmesg output). You may want to make sure not to have your boot media included there if you've booted off a usb thumb drive, for example.

Using that output, you can build a loop and iterate the drives and build your labels (or whatever).

Something like (pseudo-code):

Code: 
for DISK in $(sysctl -n kern.disks)
do
    echo "here could have been the code to create your labels on disk ${DISK} but you haven't paid for it."
done

HTH!
 
vwe@ said:
echo "here could have been the code to create your labels on disk ${DISK} but you haven't paid for it."
Click to expand...

hahhahaha =)))) good one :)))

Thx for the hint, I will most definitely try it :).
 
Just one thing with the sysctl variable; it prints the devices from the last probed one to the first probed one. Is there a way (via this sysctl option) to reverse the output ?
 
You must log in or register to reply here.
Back
Top