non bootable mirror

I've been following the guide from the Handbook to create the RAID (20.4.2 Creating a Mirror with Two New Disks) and that goes great. Nice and easy, however I can't boot off the mirror. Goes to mountroot

Code:
Mounting from ufs:/dev/mirror/gm0s1a failed with error 19.

Loader variables:
   vfs.root.mountfrom=ufs:/dev/mirror/gm0s1a
   vfs.root.mountfrom.options=rw

Listing of managed disk devices shows:
Code:
mirror/gm0 ada1 ada0
Trying to mount mirror/gm0 gives an error.

Also my testing seems to mean I have to re-install the bootcode to the mirror every time or I just look at a blinking cursor instead of getting to the FreeBSD Menu with the mountroot issue.

Code:
root@MyServer:/home/uxi # gpart bootcode -b /boot/mbr mirror/gm0
bootcode written to mirror/gm0
root@MyServer:/home/uxi # gpart set -a active -i 1 mirror/gm0
active set on mirror/gm0s1
root@MyServer:/home/uxi # gpart bootcode -b /boot/boot mirror/gm0s1
bootcode written to mirror/gm0s1

When I go back on the original ada0 drive, it seems I have to redo the first couple steps every time. At first I was scared that I lost all progress and was going to start over, but I first boot from the ada0 and load gmirror, I see nothing until

Code:
FreeBSD MyServer.mydomain.net 9.1-RELEASE FreeBSD 9.1-RELEASE #1 r251642M: Wed Jun
12 10:44:36 PDT 2013     root@MyServer.mydomain.net:/usr/obj/usr/src/sys/MYSERVERIPF amd64
root@MyServer:/home/uxi # gmirror load
root@MyServer:/home/uxi # gmirror label -v gm0 /dev/ada1 /dev/ada2
gmirror: Can't store metadata on /dev/ada1: Operation not permitted.
root@MyServer:/home/uxi # gpart create -s MBR mirror/gm0
mirror/gm0 created
root@MyServer:/home/uxi # gpart show
=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0k)
          40         128     1  freebsd-boot  (64k)
         168     8388608     2  freebsd-ufs  (4.0G)
     8388776     8388608     3  freebsd-ufs  (4.0G)
    16777384     2097152     4  freebsd-swap  (1.0G)
    18874536   954204160     5  freebsd-ufs  (455G)
   973078696  2933950439        - free -  (1.4T)

=>       63  976773104  mirror/gm0  MBR  (465G)
         63  976773104              - free -  (465G)

root@MyServer:/home/uxi # gpart add -t freebsd -a 4k mirror/gm0
mirror/gm0s1 added
root@MyServer:/home/uxi # gpart show
=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0k)
          40         128     1  freebsd-boot  (64k)
         168     8388608     2  freebsd-ufs  (4.0G)
     8388776     8388608     3  freebsd-ufs  (4.0G)
    16777384     2097152     4  freebsd-swap  (1.0G)
    18874536   954204160     5  freebsd-ufs  (455G)
   973078696  2933950439        - free -  (1.4T)

=>       63  976773104  mirror/gm0  MBR  (465G)
         63         63              - free -  (31k)
        126  976772979           1  freebsd  (465G)
  976773105         62              - free -  (31k)

=>        0  976772979  mirror/gm0s1  BSD  (465G)
          0          2                - free -  (1.0k)
          2    8388608             1  freebsd-ufs  (4.0G)
    8388610    4194304             2  freebsd-swap  (2.0G)
   12582914    8388608             4  freebsd-ufs  (4.0G)
   20971522  955801456             5  freebsd-ufs  (455G)
  976772978          1                - free -  (512B)

root@MyServer:/home/uxi # mount -rw /dev/mirror/gm0s1
gm0s1%  gm0s1a% gm0s1b% gm0s1d% gm0s1e% 
root@MyServer:/home/uxi # mount -rw /dev/mirror/gm0s1a /backup
root@MyServer:/home/uxi # mount -rw /dev/mirror/gm0s1
gm0s1%  gm0s1a% gm0s1b% gm0s1d% gm0s1e% 
root@MyServer:/home/uxi # mount -rw /dev/mirror/gm0s1d /backup/var/
root@MyServer:/home/uxi # mount -rw /dev/mirror/gm0s1e /backup/usr/
root@MyServer:/home/uxi # df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
/dev/ada0p2           3.9G    740M    2.9G    20%    /
devfs                 1.0k    1.0k      0B   100%    /dev
/dev/ada0p3           3.9G    175M    3.4G     5%    /var
/dev/ada0p5           440G    6.0G    399G     1%    /usr
/dev/mirror/gm0s1a    3.9G    1.4G    2.2G    39%    /backup
/dev/mirror/gm0s1d    3.9G    175M    3.4G     5%    /backup/var
/dev/mirror/gm0s1e    441G    6.1G    400G     2%    /backup/usr
root@MyServer:/home/uxi # 
root@MyServer:/home/uxi # gpart show
=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0k)
          40         128     1  freebsd-boot  (64k)
         168     8388608     2  freebsd-ufs  (4.0G)
     8388776     8388608     3  freebsd-ufs  (4.0G)
    16777384     2097152     4  freebsd-swap  (1.0G)
    18874536   954204160     5  freebsd-ufs  (455G)
   973078696  2933950439        - free -  (1.4T)

=>       63  976773104  mirror/gm0  MBR  (465G)
         63         63              - free -  (31k)
        126  976772979           1  freebsd  (465G)
  976773105         62              - free -  (31k)

=>        0  976772979  mirror/gm0s1  BSD  (465G)
          0          2                - free -  (1.0k)
          2    8388608             1  freebsd-ufs  (4.0G)
    8388610    4194304             2  freebsd-swap  (2.0G)
   12582914    8388608             4  freebsd-ufs  (4.0G)
   20971522  955801456             5  freebsd-ufs  (455G)
  976772978          1                - free -  (512B)
 
fstab on the ada0:

Code:
/dev/ada0p2     /               ufs     rw      1       1
/dev/ada0p3     /var            ufs     rw      2       2
/dev/ada0p4     none            swap    sw      0       0
/dev/ada0p5     /usr            ufs     rw      2       2

and the mirror:

Code:
/dev/mirror/gm0s1a      /               ufs     rw      1       1
/dev/mirror/gm0s1d      /var            ufs     rw      2       2
/dev/mirror/gm0s1b      none            swap    sw      0       0
/dev/mirror/gm0s1e      /usr            ufs     rw      2       2
 
Maybe relevant:
Code:
 # gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ada2 (ACTIVE)
                      ada1 (ACTIVE)

 # gmirror list
Geom name: gm0
State: COMPLETE
Components: 2
Balance: load
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 975885528
Providers:
1. Name: mirror/gm0
   Mediasize: 500107861504 (465G)
   Sectorsize: 512
   Mode: r3w3e9
Consumers:
1. Name: ada2
   Mediasize: 500107862016 (465G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 1
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 2140778484
2. Name: ada1
   Mediasize: 500107862016 (465G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 3626640160
 
I noticed when I'm shutting down to pull the ada0 and set the RAID as boot, I'm seeing:

Code:
GEOM_Mirror: Device gm0: provider mirror/gm0 destroyed
GEOM_MIRROR: Device gm0 destroyed.
 
Back
Top