Solved FreeBSD zfs installation script not working

fred974

fred974

Hi,
I have tried to modify a script provided by calomel.org to my need but so far I did get really far...

The script is executed from the usb key and looking at the output log, the first error message that I get is
Code: 
gpart: autofill: No space left on device

I use a dell R610 with6x 146 GB SAS disks set as 6 RAID-0 LDs in the raid controler.
Here is the script:
Code: 
#!/bin/sh
#
# Calomel.org
#     https://calomel.org/zfs_freebsd_root_install.html
#     FreeBSD 10.2-RELEASE ZFS Root Install script
#     zfs_trinity_FreeBSD10.2.sh @ Version 0.19

echo "# remove any old partitions on destination drive"
umount zroot
umount /mnt
zpool destroy zroot

gpart delete -i 2 mfid0
gpart delete -i 2 mfid1
gpart delete -i 2 mfid2
gpart delete -i 2 mfid3
gpart delete -i 2 mfid4
gpart delete -i 2 mfid5

gpart delete -i 1 mfid0
gpart delete -i 1 mfid1
gpart delete -i 1 mfid2
gpart delete -i 1 mfid3
gpart delete -i 1 mfid4
gpart delete -i 1 mfid5

gpart destroy -F mfid0
gpart destroy -F mfid1
gpart destroy -F mfid2
gpart destroy -F mfid3
gpart destroy -F mfid4
gpart destroy -F mfid5

echo ""
echo "# Create zfs boot (512k) and a 1 TB zfs root partition"
gpart create -s gpt mfid0
gpart create -s gpt mfid1
gpart create -s gpt mfid2
gpart create -s gpt mfid3
gpart create -s gpt mfid4
gpart create -s gpt mfid5

gpart add -a 4k -s 512k -t freebsd-boot mfid0
gpart add -a 4k -s 512k -t freebsd-boot mfid1
gpart add -a 4k -s 512k -t freebsd-boot mfid2
gpart add -a 4k -s 512k -t freebsd-boot mfid3
gpart add -a 4k -s 512k -t freebsd-boot mfid4
gpart add -a 4k -s 512k -t freebsd-boot mfid5

gpart add -a 4k -s 1T -t freebsd-zfs -l disk0 mfid0
gpart add -a 4k -s 1T -t freebsd-zfs -l disk1 mfid1
gpart add -a 4k -s 1T -t freebsd-zfs -l disk2 mfid2
gpart add -a 4k -s 1T -t freebsd-zfs -l disk3 mfid3
gpart add -a 4k -s 1T -t freebsd-zfs -l disk4 mfid4
gpart add -a 4k -s 1T -t freebsd-zfs -l disk5 mfid5

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid2
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid3
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid4
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 mfid5

echo ""
echo "# Align the Disks for 4K and create the pool"
gnop create -S 4096 /dev/gpt/disk0
gnop create -S 4096 /dev/gpt/disk1
gnop create -S 4096 /dev/gpt/disk2
gnop create -S 4096 /dev/gpt/disk3
gnop create -S 4096 /dev/gpt/disk4
gnop create -S 4096 /dev/gpt/disk5

zpool create -f -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache zroot raidz2 /dev/gpt/disk0.nop /dev/gpt/disk1.nop /dev/gpt/disk2.nop /dev/gpt/disk3.nop /dev/gpt/disk4.nop /dev/gpt/disk5.nop

mkdir /mnt/usr
zfs create -o setuid=off -o mountpoint=/usr/ports zroot/ports
zfs create -o exec=off -o setuid=off zroot/ports/distfiles
zfs create -o exec=off -o setuid=off zroot/ports/packages

zfs create -o mountpoint=/usr/local zroot/local

zfs create zroot/var
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o exec=on -o setuid=off zroot/var/tmp
chmod 1777 /mnt/var/tmp

zfs create zroot/home
zfs create zroot/tmp
chmod 1777 /mnt/tmp

zpool export zroot
gnop destroy /dev/gpt/disk0.nop
gnop destroy /dev/gpt/disk1.nop
gnop destroy /dev/gpt/disk2.nop
gnop destroy /dev/gpt/disk3.nop
gnop destroy /dev/gpt/disk4.nop
gnop destroy /dev/gpt/disk5.nop

zpool import -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache zroot

echo ""
echo "# Set the bootfs property and set options"
zpool set bootfs=zroot zroot
zpool set listsnapshots=on zroot
zpool set autoreplace=on zroot
#zpool set autoexpand=on zroot
zfs set checksum=fletcher4 zroot
zfs set compression=lz4 zroot
zfs set atime=off zroot
zfs set copies=3 zroot
#zfs set mountpoint=/ zroot

echo ""
echo "# Add swap space and apply options"
zfs create -V 4G zroot/swap
zfs set org.freebsd:swap=on zroot/swap

echo ""
echo "# Create a symlink to /home and fix some permissions"
cd /mnt/zroot ; ln -s usr/home home

echo ""
echo "# Install FreeBSD OS from *.txz memstick."
echo "# This will take a few minutes..."
cd /usr/freebsd-dist
export DESTDIR=/mnt/zroot

# Option 1: only install a 64bit os without 32bit libs
  for file in base.txz kernel.txz doc.txz ports.txz src.txz;

# Option 2: also include 32bit compatable libs
#for file in base.txz lib32.txz kernel.txz doc.txz ports.txz src.txz;

do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done

echo ""
echo "# Copy zpool.cache to install disk."
cp /var/tmp/zpool.cache /mnt/zroot/boot/zfs/zpool.cache

echo ""
echo "# Setup ZFS root mount and boot"
echo 'zfs_enable="YES"' >> /mnt/zroot/etc/rc.conf
echo 'zfs_load="YES"' >> /mnt/zroot/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/zroot/boot/loader.conf

echo ""
echo "# use gpt ids instead of gptids or disks idents"
echo 'kern.geom.label.disk_ident.enable="0"' >> /mnt/zroot/boot/loader.conf
echo 'kern.geom.label.gpt.enable="1"' >> /mnt/zroot/boot/loader.conf
echo 'kern.geom.label.gptid.enable="0"' >> /mnt/zroot/boot/loader.conf

echo ""
echo "# enable networking, pf and ssh and stop syslog from listening."
echo 'hostname="FreeBSD10ZFS"' >> /mnt/zroot/etc/rc.conf
echo '#ifconfig_bce0="dhcp"' >> /mnt/zroot/etc/rc.conf
echo 'ifconfig_bce0="inet 192.168.1.185 netmask 255.255.255.0 lladdr 00:11:22:33:44:55"' >> /mnt/zroot/etc/rc.conf
echo 'defaultrouter="192.168.1.1"' >> /mnt/zroot/etc/rc.conf
echo '#pf_enable="YES"' >> /mnt/zroot/etc/rc.conf
echo '#pflog_enable="YES"' >> /mnt/zroot/etc/rc.conf
echo 'sshd_enable="YES"' >> /mnt/zroot/etc/rc.conf
echo 'syslogd_flags="-ss"' >> /mnt/zroot/etc/rc.conf

echo ""
echo "# sshd, disable remote root logins."
echo 'PermitRootLogin no' >> /mnt/zroot/etc/ssh/sshd_config
echo 'PermitEmptyPasswords no' >> /mnt/zroot/etc/ssh/sshd_config
echo 'Banner /etc/ssh/ssh-banner' >> /mnt/zroot/etc/ssh/sshd_config
cat << 'EOF' > /mnt/zroot/etc/ssh/ssh-banner
*********************************************************************************
*       This system is for the use of authorized users only.  Usage of          *
*       this system may be monitored and recorded by system personnel.          *
*                                                                               *
*       Anyone using this system expressly consents to such monitoring          *
*       and is advised that if such monitoring reveals possible                 *
*       evidence of criminal activity, system personnel may provide the         *
*       evidence from such monitoring to law enforcement officials.             *
*                                                                               *
*********************************************************************************
EOF

echo ""
echo "# /etc/rc.conf disable sendmail"
echo 'dumpdev="NO"' >> /mnt/zroot/etc/rc.conf
echo 'sendmail_enable="NO"' >> /mnt/zroot/etc/rc.conf
echo 'sendmail_submit_enable="NO"' >> /mnt/zroot/etc/rc.conf
echo 'sendmail_outbound_enable="NO"' >> /mnt/zroot/etc/rc.conf
echo 'sendmail_msp_queue_enable="NO"' >> /mnt/zroot/etc/rc.conf

echo ""
echo "# touch the /etc/fstab else freebsd will not boot properly"
touch /mnt/zroot/etc/fstab

sync
echo ""
echo "# Syncing... Install Done."
echo ""
echo "# Hint: poweroff, remove the USB drive and re-boot the machine."
echo "#       Then add a privlidged user to the 'wheel' group. You will"
echo "#       then be able to ssh in as the new user and configure the box."
echo ""
sync

#### EOF ####

Hope somone can help.
 
Are the disks bigger than 1 TB? If you just want to use all the remaining space for ZFS, I would remove the 1T size from the gpart command and just let it automatically use the remaining space.

On recent FreeBSD versions you shouldn't need to mess with gnop. You can just set vfs.zfs.min_auto_ashift=12 with sysctl before creating the pool to force it to use 4k blocks.

Personally I'm not a fan of using the root dataset for the root file system, especially if it's not an independent pool being used exclusively for the os. I think it's messy. You're better off using pool/ROOT/somename. It then also works with the beadm boot environments system.

Also did you add the copies=3? Storing 3 copies of all data on a pool that already has double redundancy seems a bit excessive, even to me.

Plus be aware autoreplace doesn't really do anything on FreeBSD at the moment. (I say at the moment but we've been saying that for years now)
 
usdmatt said:
Are the disks bigger than 1 TB
Click to expand...
No I have 6 SAS disk of 146GB each. If am righth thats total of 584GB in total..
So are you saying that I should replace
Code: 
gpart add -a 4k -s 1T -t freebsd-zfs -l disk0 mfid0
gpart add -a 4k -s 1T -t freebsd-zfs -l disk1 mfid1
gpart add -a 4k -s 1T -t freebsd-zfs -l disk2 mfid2
gpart add -a 4k -s 1T -t freebsd-zfs -l disk3 mfid3
gpart add -a 4k -s 1T -t freebsd-zfs -l disk4 mfid4
gpart add -a 4k -s 1T -t freebsd-zfs -l disk5 mfid5
with
Code: 
gpart add -a 4k -s -t freebsd-zfs -l disk0 mfid0
gpart add -a 4k -s -t freebsd-zfs -l disk1 mfid1
gpart add -a 4k -s -t freebsd-zfs -l disk2 mfid2
gpart add -a 4k -s -t freebsd-zfs -l disk3 mfid3
gpart add -a 4k -s -t freebsd-zfs -l disk4 mfid4
gpart add -a 4k -s -t freebsd-zfs -l disk5 mfid5
Will this fix my error message?
Sorry but could you please show me how to change the code so it set vfs.zfs.min_auto_ashift=12
 
Yeah trying to create a 1T partition on a 146G disk probably isn't going to work.
You need to get rid of the -s as well. The partition will just automatically use all remaining space on the disk.

You should be able to get rid of all the gnop(8) stuff and just use
Code: 
sysctl vfs.zfs.min_auto_ashift=12
You may want to try running that manually first as I'm sat on the sofa and haven't actually tested it.
 
You must log in or register to reply here.
Back
Top