struggling to mirror mbr root pool after install

tangles

tangles

Hi,

I have an old intel board (48DXBT2) that refuses to boot GPT (UEFI enabled = fail with GPT too) and after many attempts to use GPT, I've only successfully booted/installed with GRUB and MBR.
Consequently, I've used PC-BSD 10.0 ISO (choosing server option) to do the install as it installs GRUB by default.

So, after install, I can log in as root, and have:

Code: 
# zpool status
  pool: tank
 state: ONLINE
  scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  ada0s1a   ONLINE       0     0     0

errors: No known data errors
#

Code: 
# gpart show
=>       63  117231345  ada0  MBR  (56G)
         63         63        - free -  (32K)
        126  117231219     1  freebsd  [active]  (56G)
  117231345         63        - free -  (32K)

=>        0  117231219  ada0s1  BSD  (56G)
          0  113113088       1  freebsd-zfs  (54G)
  113113088    4118131          - free -  (2.0G)
#

I'm not exactly sure why there's 2GB free... I think it's a resultant from previous install attemtps where the 2GB would've been used for swap. I don't mind though, because I hear it's good to not use all the disk in case you have to replace a disk and therefore avoid potential size-matching issues...

Continuing on:

Code: 
# camcontrol devlist
<KINGSTON SV300S37A60G 527ABBF0>   at scbus3 target 0 lun 0 (ada0,pass0)
<KINGSTON SV300S37A60G 527ABBF0>   at scbus4 target 0 lun 0 (ada1,pass1)
<AHCI SGPIO Enclosure 1.00 0001>   at scbus9 target 0 lun 0 (pass2,ses0)
#

and

Code: 
# ll /dev/ada*
crw-r-----  1 root  operator  0x72 May 31 05:17 /dev/ada0
crw-r-----  1 root  operator  0x77 May 31 05:17 /dev/ada0s1
crw-r-----  1 root  operator  0x7d May 31 05:17 /dev/ada0s1a
crw-r-----  1 root  operator  0x74 May 31 07:32 /dev/ada1
#

So now I want to mirror ada0 with ada1, but where I'm struggling is how to setup ada1's boot code and partitions to be identical to ada0, before I use the zpool attach command.

I've read up on booting at http://www.schmidp.com/2014/01/05/how-f ... ts-on-zfs/ but I just don't know enough about the boot process to understand what to do next because I don't know if PC-BSD has used /boot/boot0, /boot/mbr or /boot/pmbr to setup booting.

(I'm so close to chucking this mobo to the shitter, and bags a UEFI capable board... but thats > $300 :( )

I've already failed in attempting to setup the partitions/slices on ada1 and so if anyone can assist here, it would be much appreciated.
 
A UEFI motherboard is not needed to use GPT. Any normal BIOS machine where the vendor didn't try to do something "smart" should boot from a GPT disk with a PMBR.

Please back up anything important first.

Important: these drives are SSDs. It's important to use 4K blocks on them for write performance, and also to make sure that the 4K filesystem blocks align with the 4K drive blocks. So use gnop(8) to create a 4K-block fake device for the first, to force ZFS to ashift=12. The second is done on MBR by aligning the FreeBSD "bsdlabel" partition when creating it ( gpart add -t freebsd-ufs -a4k ada0s1). If you did not do these steps already, go back and do them now while it's still easy. Reference: Disk Setup On FreeBSD.

(Untested.)
Duplicate the MBR partitioning from ada0 to ada1:
gpart backup ada0 | gpart restore -F ada1

Copy the BSD partitioning inside the MBR partitioning from ada0s1 to ada1s1:
gpart backup ada0s1 | gpart restore -F ada1s1

Install bootcode on ada1. This is that nasty mess of things needed for ZFS on MBR, including dd(1). The mirror will work without it, but if ada0 failed, ada1 would not be able to boot because the bootcode has not been installed to it.

Finally, add /dev/ada1s1a to the ZFS mirror.
 
HI wblock,

Well I definitely have a motherboard that's "too smart" for it's own good.

I used FreeBSD 10 on a usb stick to perform another install, and I chose ZFS on Root with GPT for the disks. I can see it's aligning the partitions/slices as you have recommended and the install goes smoothly.

All I get on screen after restart is " No bootable device -- insert boot disk and press any key"

Just so you know, I've also chosen ZFS on Root with MBR for the disks. It fails too, but when I inspect the disks on a Mac, Disk Utility.app shows that they are GPT formatted still... ??

So clearly my intel 48DXBT2 has issues reading how FreeBSD 10 conifgures drives to boot.

I also have an X58A-UD7 in another case. Connecting up the two SSDs (ZFS on Root using GPT) on this motherboard boot up perfectly fine.

Would it be possible for me to use the shell right at the end of the install and issue bootcode commands to get the intel 48DXBT2 to see the disks?
 
I think we're going back and forth. If the motherboard can't use GPT, well it's not good, but MBR is livable. But there are some other things that can be tried with GPT, also.

One motherboard works with GPT, the other--with the exact same disks and install--does not. Correct?
 
That's correct.
FreeBSD 10.0 via CDROM or USB image:
1. Gigabyte X58AUD7. --> ZFS on Root with GPT Mirror (and aligned) = bootable

2. Intel DX48BT2. --> ZFS on Root with GPT Mirror (and aligned) = fail

3. Intel DX48BT2. --> ZFS on Root with MBR Mirror = fail

PC-BSD via CDROM: ( it uses GRUB by default)
4. Intel DX48BT2. --> ZFS on Root with GPT Mirror (and aligned) = fail

5. Intel DX48BT2. --> ZFS on Root with MBR using TrueOS install non-Mirror = success!

This is why my first post was asking how to attach mirror and do the necessary boot code stuff... Cos No 5. Is the only way (so far) that I can even boot from a ZFS install with this mobo...

Thank you for persevering with this wblock,
 
Maybe I should see if I can get my hand on the PC-BSD TrueOS installer script, to see how it is configuring successfully...
 
found the install log for PC-BSD.
It's located: /root/pc-sysinstall.log

Now that I know how the disk was prepared, I should be right from here on to drop to the shell and follow the log along and factor in the mirror.

I sure have learnt a lot having a "finicky" mobo...

Here's the relevant bootcode output:

Code: 
kern.geom.debugflags: 0 -> 16
Deleting all gparts
Running: gpart delete -i 1 /dev/ada0
ada0s1 deleted
Running: gpart destroy /dev/ada0
ada0 destroyed
Clearing gpt backup table location on disk
Running: dd if=/dev/zero of=/dev/ada0 bs=1m count=1
1+0 records in
1+0 records out
1048576 bytes transferred in 0.017103 secs (61308778 bytes/sec)
Running: dd if=/dev/zero of=/dev/ada0 bs=1m oseek=57237
dd: /dev/ada0: short write on character device
dd: /dev/ada0: end of device
5+0 records in
4+1 records out
5136384 bytes transferred in 0.031866 secs (161186598 bytes/sec)
Running: dd if=/dev/zero of=/dev/ada0 count=3000
3000+0 records in
3000+0 records out
1536000 bytes transferred in 0.181059 secs (8483428 bytes/sec)
Running gpart on /dev/ada0
Running: gpart create -s mbr -f active /dev/ada0
ada0 created
Running gpart add on /dev/ada0
Running: gpart add -a 4k -t freebsd -i 1 /dev/ada0
ada0s1 added
Cleaning up /dev/ada0s1
Running: dd if=/dev/zero of=/dev/ada0s1 count=1024
1024+0 records in
1024+0 records out
524288 bytes transferred in 0.074441 secs (7043005 bytes/sec)
Running: gpart set -a active -i 1 /dev/ada0
active set on ada0s1
Running: gpart create -s BSD /dev/ada0s1
ada0s1 created
Running: gpart add -s 55231M -t freebsd-zfs -i 1 /dev/ada0s1
ada0s1a added
Running: gpart bootcode -b /boot/boot /dev/ada0s1
bootcode written to ada0s1

Now to start again and see if it works, as it doesn't look like any alignment was done...
 
It did try to set alignment, but not where it will help. And there is an off-by-one error when it overwrites that backup GPT.

The call to use alignment when creating an MBR slice is ignored by gpart(8). However, BSD partitions can be aligned when added.
 
Dropping to the shell and attempting to manually set up the pool didn't work for me.

By doing so, I think things like taking care of zpool cache got left out and grub didn't seem to get installed either...

In the end, I just created a pool with one drive using TrueOS install and let that complete.
I then used the gpart backup and restore suggestion you mentioned earlier to prepare the ada1 disk.

I then attached the drive to the pool and then manually installed the bootcode and Grub.

That worked! I've tested to by removing each disk and the mobo successfully boots on either disk on any sata port.

I know the pool is not aligned cos zdb says ashift=9... I'm not all that bothered by it given that it's only the boot pool and is only going to be sharing files here at home from another pool that is aligned...

Thanks again wblock! As I wouldn't have got as far as I have without your help.

Maybe some time soon I'll see if I can get freebsd 10 installed and aligned on another pair of SSDs, seeing though I now know how to boot this damn mobo now!

I'll see if I can adopt http://daemon-notes.com/articles/system/install-zfs/begin to my needs which will involve including grub-install --force /dev/ada0/1 while in chroot, oh and grub doesn't like mbr labels either..

Thanks.
 
You must log in or register to reply here.
Back
Top