Clone a zfs disk

rdunkle · Oct 8, 2016

This is a step by step method using zfs send to clone a FreeBSD 10.3 ZFS disk.
The target disk obviously needs to be at least large enough to fit the
source image. Adjustments can be made to partitions on the target disk.

We will use two machines. The source and the target machines.
The target machine will use a USB stick to boot the MFS version of FreeBSD.
The MFS FreeBSD is a very useful rescue tool.

Create the mfs FreeBSD usb stick:

Download mfs FreeBSD
The usb stick can be created using the source machine or any other FreeBSD machine available.
Download the appropriate image file.
http://mfsbsd.vx.sk/files/images/10/amd64/

mfsbsd-10.3-RELEASE-amd64.img
Do a check sum to verify image:

 md5 mfsbsd-10.3-RELEASE-amd64.img

Code:

MD5 (mfsbsd-10.3-RELEASE-amd64.img) = 7f80e681e378eb3dfbb7d8b565245916

Root password for all MFS images: mfsroot
All images have mfsbsd.autodhcp set - all network cards are configured for DHCP.

On the source machine you can usually find the device name of the the usb stick
by plugging in the stick and watching /var/log/messages.
Watch for something like this:

Code:

da0 at umass-sim0 bus 0 scbus2 target 0 lun 0

That tells us device name: da0

Write img file to usb stick

 dd if=mfsbsd-10.3-RELEASE-amd64.img  of=/dev/da0 bs=1M conv=sync

Code:

52+1 records in
53+0 records out
55574528 bytes transferred in 0.860368 secs (64593903 bytes/sec)

Boot the target system using the usb stick.
Use the Boot Multi user option (the default)
Then find out the ip address of the target system. This is needed
to allow the ssh connection from source and target.
The target system will use dhcp and have the hostname: mfsbsd

On the source machine:
Check the stats on the source hard drive:

 zpool status

Code:

  pool: zroot
 state: ONLINE
  scan: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        zroot          ONLINE       0     0     0
          gpt/system0  ONLINE       0     0     0

errors: No known data errors

* Notice the gpt/system0 this will be needed to create the
* target disk.

Command: zpool list zroot -- this gives us some idea of capacity
zpool list zroot

Code:

NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot  2.70T   539G  2.18T         -     9%    19%  1.00x  ONLINE  -

In this case we could even use a smaller disk on the target since we see 19% capacity.
--------------------
On the source machine
Check the gpt layout of the source disk.
We will want to use similar layout and adjust as necessary for size of target disk.

Source machine:
gpart show

Code:

=>        34  5860533101  ada0  GPT  (2.7T)
          34           6        - free -  (3.0K)
          40        1024     1  freebsd-boot  (512K)
        1064  5825888256     2  freebsd-zfs  (2.7T)
  5825889320    29360128     3  freebsd-swap  (14G)
  5855249448     5283687        - free -  (2.5G)

Add mfsbsd to /etc/hosts on the source machine for ssh connection use
* Each time mfsbsd boots there is a new key for ssh generated

From the source machine:
login to target (mfsbsd system):

 ssh -l root mfsbsd

*(Edit your .ssh/known_hosts as needed)

On target system load zfs and set the ashift value:
kldload zfs
sysctl vfs.zfs.min_auto_ashift=12
Use dmesg to find device name for hard drive

In this example we see:

Code:

ada0: <TOSHIBA DT01ACA200 MX4OABB0> ATA8-ACS SATA 3.x device

ada0 is the device name
For purposes of this example we assume the target disk is already wiped.
This example also assumes to use a different target disk size.
In this example the target disk is smaller. GPT slices will need adjustments.
Create gpt table on target disk:

gpart create -s gpt ada0

Code:

ada0 created

Create boot slice on the target disk:

gpart add -s 1024 -t freebsd-boot ada0

Code:

=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0K)
          40        1024     1  freebsd-boot  (512K)
        1064  3907028071        - free -  (1.8T)

We want to leave extra free space and 16 GB for swap slice
Very rough math:
1.8 TB = 1800 GB
In this example we create a 1830GB slice for the FreeBSD:

gpart add -s 1830G -t freebsd-zfs -l system0 ada0

Check results
gpart show

Code:

=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0K)
          40        1024     1  freebsd-boot  (512K)
        1064  3837788160     2  freebsd-zfs  (1.8T)
  3837789224    69239911        - free -  (33G)

Now add the swap partition.
Make sure the name is the same as the name on the source disk.
Check your source disk for the swap file name:
cat /etc/fstab

Code:

# Device        Mountpoint   FStype  Options  Dump  Pass#
/dev/gpt/swap0  none  swap    sw  0  0

This command will add the 16GB swap slice:
gpart add -s 16G -t freebsd-swap -l swap0 ada0

Check the results:
gpart show

Code:

=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0K)
          40        1024     1  freebsd-boot  (512K)
        1064  3837788160     2  freebsd-zfs  (1.8T)
  3837789224    33554432     3  freebsd-swap  (16G)
  3871343656    35685479        - free -  (17G)

*
* If you make mistakes on create sizes this is an example of the command
* to remove slice 2:
* gpart delete -i2 ada0
*
Finished with gpart

Time to create the zpool on the target disk.
zpool create -f -o altroot=/mnt -O canmount=off -m none zroot /dev/gpt/system0

Now a couple of useful settings on the target disk zfs pool:
zfs set checksum=fletcher4 zroot
zfs set atime=off zroot

Let's double check ashift is correct:
zdb -C -e -p /dev/gpt zroot | grep ashift

Code:

ashift: 12

12 means --> 4k alignment
Copy bootcode to target disk:
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
Next step is on the source system.
Login as root

Take zfs snapshot:
zfs snapshot -r zroot@backup

Send snapshot to other system:
zfs send -R zroot@backup | ssh root@mfsbsd zfs recv -F zroot

Code:

The authenticity of host 'mfsbsd (192.168.2.104)' can't be established.
ECDSA key fingerprint is SHA256:j0pjqX2Sjce8eZgLJVIJX5BgTUGIk7mrOUE3OL7Mp3k.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mfsbsd' (ECDSA) to the list of known hosts.
Password for root@mfsbsd:

The data transfer starts !
Be patient this depends on network and system speed

You can monitor the transfer on the target system:

zpool list zroot

Code:

NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot  1.78T  3.22G  1.78T         -     0%     0%  1.00x  ONLINE  /mnt

Watch the SIZE grow

Steps after send completes:
A little housekeeping on the target system
zpool set bootfs=zroot zroot
zpool export zroot
zpool import zroot
Shutdown the target system
On the source system you may want to destroy the snapshot:
zfs destroy -rv zroot@backup
Clone process complete

wblock@ · Oct 9, 2016

rdunkle said:
zfs set checksum=fletcher4 zroot

This is the default, probably best to leave it alone.

ucomp · Dec 7, 2017

rdunkle said:
zpool set bootfs=zroot zroot

that interrupted the boot process at stage two and brought me to boot-prompt .
no way out from boot: ( can't load /boot/kernel/kernel)

until I realized ( on installed target-disk(or target-machine)) :
zpool get bootfs
NAME PROPERTY VALUE SOURCE
zroot bootfs zroot/ROOT/default local

zpool set bootfs=zroot/ROOT/default zroot
fixed it.

I would not touch 'set bootfs' again

easyiest way to clone disk on local machine(without ssh) I found out :
Install bootable root-zfs from install-DVD to your clone-target - disk
(name the zroot whatever you want: e.g zroot2)

boot the newly installed system
zpool export zroot2

now boot back to your source-disk
zpool import zroot2
zfs snapshot -r zroot@whateveryouwant
zfs send -R zroot@whateveryouwant | zfs recv -F zroot2

now boot into your cloned disk.

you can do the same with ssh: Install from DVD to target and send/recv via ssh
( maybe I forgot 1 or 2 steps, i`ve written this from memory in tired state)

I was very surprised how easy it is(after all that headache) to clone a bootable zfs-root-disk, it`s only not well documented.

wblock@ said:
This is the default, probably best to leave it alone.

that's the nice thing with DVD-installation: you do not touch the reasonable defaults and don`t have to think about them too much
( in my case with the exception of renaming zroot to zroot2 for not to confuse pool-naming)

Clone a zfs disk

rdunkle

wblock@

ucomp