Cloned ZFS drive won't boot (zfs send & recv)

sim · #1 February 8th, 2011, 14:22

Hi

I have a fully working installation of FreeBSD 8-STABLE (gpt root on ZFS). I want to clone this system onto another drive so I can put it in a second machine. This is what I did:

Using the standard root-on-ZFS technique, I:

created an MBR on the new drive,
set up partitions using gpart (freebsd-zfs: ad12s1a; freebsd-swap ad12s1b),
dd'd the gptbootzfs file to ad12s1 and ad12s1a as usual
created a simple pool in ad12s1a (zsystem)
copied my working pool (zroot) to the new one: zfs send -R zroot@freeze | zfs recv -Fdv zsystem
checked that the new zsystem filesystems were present using zfs list.

All this went really well. But when I try to boot from the new ad12 drive, it just blocks immediately after the 'F1 FreeBSD' boot partition prompt.

Does anyone have any ideas I can try? Did I miss something out?

Many thanks

sim

jem · #2 February 8th, 2011, 14:31

You seem to have gotten confused between using MBR and GPT partitioning.

You have set up an MBR partition containing a bsdlabel on your second disk (ad12s1a), which is entirely different from the GPT setup on your first disk.

When you use 'gpart', ensure you specify the 'GPT' scheme when you label the disk.

sim · #3 February 8th, 2011, 15:55

Ah, hmm, ok I'll have to check (am at work atm). I used this guide here http://www.rhyous.com/2009/12/01/how...fs-partitions/ up to step 19, which is where I did the zfs send/recv.

Thanks

sim

jem · #4 February 8th, 2011, 16:14

From your original post, it seems that your existing setup is a ZFS on GPT configuration. If you want to move it to another disk, you need to create a ZFS on GPT configuration on the new disk too (unless you deliberately want to convert to the other method).

That guide you followed assumes you're using a ZFS on MBR configuration, so the instructions aren't quite right for your situation.

If you provide the following information, I can try to show you the different commands you need to use.

Code:

# diskinfo -v ad12
# diskinfo -v (your existing disk, ad0? ad4?)
# gpart show
# zpool status -v
# zfs list

If you paste it here, please enclose it in [CODE] tags.

sim · #5 February 9th, 2011, 08:19

Hi again, I was wrong about the donor system, it's using MBR after all. Here's the info you asked for:

Donor system: 'nostromo', ad10
New system: ad12

Code:

root@nostromo> diskinfo -v ad10
ad10
        512             # sectorsize
        500107862016    # mediasize in bytes (466G)
        976773168       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        969021          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        WD-WMAV90104509 # Disk ident.

root@nostromo> diskinfo -v ad12
ad12
        512             # sectorsize
        160000000000    # mediasize in bytes (149G)
        312500000       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        310019          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        S0DFJ1HL665302  # Disk ident.

Code:

root@nostromo> gpart show
=>       63  976773105  ad10  MBR  (466G)
         63  419441022     1  freebsd  [active]  (200G)
  419441085  557332083        - free -  (266G)

=>        0  419441022  ad10s1  BSD  (200G)
          0  411052414       1  freebsd-zfs  (196G)
  411052414    8388608       2  freebsd-swap  (4.0G)

=>       63  312499908  ad12  MBR  (149G)
         63  156249954     1  freebsd  [active]  (75G)
  156250017  156249954     2  freebsd  (75G)

=>        0  156249954  ad12s2  BSD  (75G)
          0  156249954          - free -  (75G)

=>        0  156249954  ad12s1  BSD  (75G)
          0  147861346       1  freebsd-zfs  (71G)
  147861346    8388608       2  freebsd-swap  (4.0G)

zpool status - a couple of fishy things here. pool area51a is an old damaged pool, I should remove all trace of it. Also the donor pool appear to be of an older version - I wonder if I should upgrade it before doing the send/recv?:

Code:

root@nostromo> zpool status -v
  pool: area51a
 state: UNAVAIL
status: One or more devices could not be opened.  There are insufficient
        replicas for the pool to continue functioning.
action: Attach the missing device and online it using 'zpool online'.
   see: http://www.sun.com/msg/ZFS-8000-3C
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        area51a     UNAVAIL      0     0     0  insufficient replicas
          ad14s1a   UNAVAIL      0     0     0  cannot open

  pool: zroot
 state: ONLINE
status: The pool is formatted using an older on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on older software versions.
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          ad10s1a   ONLINE       0     0     0

errors: No known data errors

  pool: zsystem
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        zsystem     ONLINE       0     0     0
          ad12s1a   ONLINE       0     0     0

errors: No known data errors

zfs list after send/recv (not zroot/home & zroot/tmp were not copied due to size constraints):

Code:

NAME                          USED  AVAIL  REFER  MOUNTPOINT
zroot                         120G  72.8G   671M  legacy
zroot/home                   68.5G  72.8G  68.5G  /home
zroot/tmp                    7.82G  72.8G  7.82G  /tmp
zroot/usr                    42.8G  72.8G  15.7G  /usr
zroot/usr/ports              26.8G  72.8G  13.2G  /usr/ports
zroot/usr/ports/distfiles    13.6G  72.8G  13.6G  /usr/ports/distfiles
zroot/usr/ports/packages       18K  72.8G    18K  /usr/ports/packages
zroot/usr/src                 312M  72.8G   312M  /usr/src
zroot/var                     351M  72.8G   345M  /var
zsystem                      43.7G  25.7G   671M  legacy
zsystem/usr                  42.7G  25.7G  15.7G  /usr
zsystem/usr/ports            26.7G  25.7G  13.1G  /usr/ports
zsystem/usr/ports/distfiles  13.5G  25.7G  13.5G  /usr/ports/distfiles
zsystem/usr/ports/packages     20K  25.7G    20K  /usr/ports/packages
zsystem/usr/src               312M  25.7G   312M  /usr/src
zsystem/var                   345M  25.7G   345M  /var

I hope that's useful - thanks for your time.

sim

jem · #6 February 9th, 2011, 10:19

Sorry, I'm not overly familiar with ZFS-on-MBR setups - I've only played with ZFS-on-GPT. All I would say is that the gptzfsboot image is not what you need to dd to the partition. You'll want zfsboot instead.

I'd also suggest that if your server is running FreeBSD exclusively (i.e. not dual booting with Windows), then using GPT partitioning instead of MBR may be more straightforward for you - there's less "nesting" of partition schemes (e.g. bsdlabels inside MBR slices) with GPT . You simply create three GPT partitions at the top level of your disk, one for bootcode, one for swap and one for ZFS, create your zpool and proceed from there.

sim · #7 February 9th, 2011, 16:27

Thanks Jem - both of those observations sound reasonable. I'll have a go at recovering the existing MBR copy by trying zfsboot, if that fails I'll wipe it and try a GPT set-up.

Thanks for your input!

sim (a fellow Surrey-ite

)

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Solved] ZFS cannot replace failed drive	rbauer_snow	Storage	6	March 24th, 2012 20:46
ZFS mirror boot: cannot boot from 2nd disk	ScottJ97	General	24	January 31st, 2011 01:17
increase zfs send/receive throughput from source to destination	ytotk	Networking	7	October 8th, 2010 06:14
Zfs root & boot	singularity	General	12	October 2nd, 2010 10:35
ZFS failed drive recovery	budric	General	6	January 4th, 2009 19:30