Add a second zfs boot disk after install

pos

pos

Hi

I have a firewall with a 32 GB msata disk. I have another and new disk on the shelf that I will add to the firewall.
I want this to be a 100% moirrored disk that I can boot from by just changing boot order in bios.


This is what it looks like before I have added the second msata....

Code: 
13:07:26 xanadu:~ # gpart show -l
=>      40  61865904  ada0  GPT  (30G)
        40      1024     1  gptboot0  (512K)
      1064       984        - free -  (492K)
      2048   4194304     2  swap0  (2.0G)
   4196352  57667584     3  zfs0  (27G)
  61863936      2008        - free -  (1.0M)


13:05:48 xanadu:~ # zpool list -v
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot         27G  5.38G  21.6G        -         -     7%    19%  1.00x  ONLINE  -
  ada0p3      27G  5.38G  21.6G        -         -     7%    19%
13:06:30 xanadu:~ #



Now lets say I have added the new second disk...... I need some help with the steps to avoid reinstall :)
I have never done this with zfs+FreeBSD on the root disk and just want to be careful...


# 1 # - Clone partition table
I assume I first have to clone my partition table. According to the man page of gpart I could do it with:
/sbin/gpart backup ada0 | /sbin/gpart restore -F ada1

# 2 # - Set up mirror
I guess this will work directly, but I cannot boot on the new disk if trying as I have no boot block. Right?
Will this use just partition 3 and leave the others (i.e not destroying my other partitions)? I mean... If the
second disk would have been blank, it would have created the partion with this command.
zpool attach zroot ada0 ada1

# 3 # - Make new disk bootable
Well... what is the proper way for me doing this based on my printed disk data above?
gpart bootcode <what...> ada1


# 4 # - swap
will the swap work out of the box if booting on the second disk as the partiton is cloned? Or does it need any other prepp?

# 5 # - Make second disk the second boot option in bios



Any pro:s that can help me avoid a disk recovery caused by me doing the wrong things :)

Tnx in advance
/Peo
 
1) Don't clone them exactly, the filesystem labels will clash (two partitions with the same label, one on each disk). It'll be better if you use an intermediate file; gpart backup ada0 > part-table.txt; then check and modify where needed before restoring to the other disk.

3) Same command as for one disk; /sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1
 
  • Thanks
Reactions: pos
SirDice said:
1) Don't clone them exactly, the filesystem labels will clash (two partitions with the same label, one on each disk). It'll be better if you use an intermediate file; gpart backup ada0 > part-table.txt; then check and modify where needed before restoring to the other disk.

3) Same command as for one disk; /sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1
Click to expand...

Thanks. I will do a backup and then import to the new disk. And I will rewrite the labels in between...

Apart from what you already have answered, is there any other major thing I need to think of? I mean, anything I have not asked due to lack of knowledge of the FreeBSD+bootzfs+mirror ?


PS
Should the bootcode index be "1" on disk 2?

Tnx
/Peo
 
pos said:
Should the bootcode index be "1" on disk 2?
Click to expand...
Look at the index of the freebsd-boot partition, that's where it needs to go. Typically it has index 1 but this doesn't have to be the case. The 'index', in this case, refers to the numbering of the partitions on the disk, not the boot order or something similar.

In my case for example, the freebsd-boot partition has index 2, because it's the second partition on disk.
Code: 
dice@williscorto:~ % gpart show
=>       40  625142368  ada0  GPT  (298G)
         40     409600     1  efi  (200M)
     409640       1024     2  freebsd-boot  (512K)
     410664        984        - free -  (492K)
     411648   16777216     3  freebsd-swap  (8.0G)
   17188864  607952896     4  freebsd-zfs  (290G)
  625141760        648        - free -  (324K)
 
  • Thanks
Reactions: pos
SirDice said:
Look at the index of the freebsd-boot partition, that's where it needs to go. Typically it has index 1 but this doesn't have to be the case. The 'index' refers to the numbering of the partitions, not the boot order or something similar.
Click to expand...

Ahhh, yes... Of course :) It´s the partition number for freebsd-boot. Now it is 1. And if I clone the disk it will be the same. I.e 1.

Tnx
 
I have now added the second disk and done the following, and it at least seems ok....


gpart backup ada0 > /root/gpart-backup-ada0-part-table.txt
vi /root/gpart-backup-ada0-part-table.txt (change lables names from <name>0 to <name>1. Maybw not needed)
gpart restore -l -F ada1 < /root/gpart-backup-ada0-part-table.txt


Code: 
18:17:59 xanadu:~ # gpart show -l
=>      40  61865904  ada0  GPT  (30G)
        40      1024     1  gptboot0  (512K)
      1064       984        - free -  (492K)
      2048   4194304     2  swap0  (2.0G)
   4196352  57667584     3  zfs0  (27G)
  61863936      2008        - free -  (1.0M)

=>      40  61865904  ada1  GPT  (30G)
        40      1024     1  gptboot1  (512K)
      1064       984        - free -  (492K)
      2048   4194304     2  swap1  (2.0G)
   4196352  57667584     3  zfs1  (27G)
  61863936      2008        - free -  (1.0M)



18:05:39 xanadu:~ # / sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1
partcode written to ada1p1




18:10:01 xanadu:~ # zpool attach zroot ada0p3 ada1p3
Make sure to wait until resilver is done before rebooting.

If you boot from pool 'zroot', you may need to update
boot code on newly attached disk 'ada1p3'.

Assuming you use GPT partitioning and 'da0' is your new boot disk
you may use the following command:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

18:10:37 xanadu:~ #

Code: 
18:14:35 xanadu:~ # zpool list -v
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot         27G  5.38G  21.6G        -         -     7%    19%  1.00x  ONLINE  -
  mirror      27G  5.38G  21.6G        -         -     7%    19%
    ada0p3      -      -      -        -         -      -      -
    ada1p3      -      -      -        -         -      -      -
18:14:39 xanadu:~ #
18:14:39 xanadu:~ #
18:14:40 xanadu:~ # zpool status
  pool: zroot
state: ONLINE
  scan: resilvered 5.38G in 0 days 00:02:02 with 0 errors on Mon Jul 15 18:12:38 2019
config:

    NAME        STATE     READ WRITE CKSUM
    zroot       ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        ada0p3  ONLINE       0     0     0
        ada1p3  ONLINE       0     0     0

errors: No known data errors
18:14:41 xanadu:~ #


Would this be ok and safe to reboot on the other or any of these two disks now?

Note that the zpool attach suggests "-b /boot/pmbr" when using the gpart bootcode command. Maybe this is not applicable here? Or is it?

I guess I have to also add "-b /boot/pmbr" as I have a GPT partitioning scheme and boot with bios that if I do not have the wrong info has the following boot sequence...
BIOS -> /boot/pmbr -> /boot/gptboot (or /boot/gptzfsboot) -> /boot/loader (or /boot/zfsloader) -> FreeBSD Kernel

Please enlighten me...

/Peo
 
To close the thread and maybe help others...


I did exactly as above, but used
Code: 
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
instead (i.e added -b) as I use bios boot.

Works flawlessly to boot on any of the disks.
So... SOLVED ! :)
 
My well respected gents and gals.

May I please necro this thread to ventilate some frustration?

I switched to ZFS after hearing it is the end al be all and look no further.

And in most ways but a few I found it actually is just that.

WHy ohh why is the following cli command missing;

zpool attach --deep-clone myRootOnZFSpool /myOnlyWorkingDiskLefgtOnMyRootPool /dev//{uninitialized disk}

and bob is your uncle ;)

But no. I have to sit here in fear wether or not I can keep my system upright or not for god knows how long while I am frantically copying files off the system ;)

I mean no disrespect crew. I have lots for all that is being done here and the whole os and its community.

I just saw an opening to show that ZFS is not dummy proof yet ;(
 
diversity said:
I switched to ZFS after hearing it is the end al be all and look no further.
Click to expand...
Where did you hear such nonsense? :)
diversity said:
WHy ohh why is the following cli command missing;
Click to expand...
Because there are proper tools to re-create the partitioning table ( gpart) (that's what you expect that switch to do, right?) and there is no need to go linux way of turning utilities into some kind of magic tools doing everything (but badly).
diversity said:
I just saw an opening to show that ZFS is not dummy proof yet ;(
Click to expand...
And it will not become one.
diversity said:
But no. I have to sit here in fear wether or not I can keep my system upright or not for god knows how long while I am frantically copying files off the system ;)
Click to expand...
Can't parse this, what exactly are you doing?
 
yuripv79 said:
Can't parse this, what exactly are you doing?
Click to expand...
Leaving Microsoft behind. Looking for a similar experiance but then ethical.

I never meant to go hardcore into bits and bobs and cubits and such.

I just care about my date is all and not handing it over to google or some other cloud guru.

Anyway, finally the resilvering is done.

pool: rpool
state: ONLINE
scan: resilvered 118G in 00:27:05 with 0 errors on Sun Oct 22 18:55:36 2023
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
/dev/disk/by-id/nvme-Samsung_SSD_980_1TB_S649NX0RC30726D-part3 ONLINE 0 0 0
/dev/disk/by-id/nvme-Samsung_SSD_980_1TB_S649NX0RB13366M-part3 ONLINE 0 0 0
/dev/sdb1 ONLINE 0 0 0

errors: No known data errors

this disk is nasty
/dev/disk/by-id/nvme-Samsung_SSD_980_1TB_S649NX0RB13366M-part3 ONLINE

But you know what. while ranting as loud as I have just now.

Thanks for allowing me btw.

I just realized the I also have a bpool. guess what??? boot pool.

I still rest my case. this is far too convoluted for a simpleton like me.

Again, no disrespect meant.

Lets see if I can system upright as I am about to try and increase it's size.
1697995222344.png


the whole thing that got me in this pickle was following a root on zfs guide somewhere and then I romantically thought. Yeah sure 120GB+ is enough for the OS, the rest we deal with later.

But now my OS is showiong signs of near death because it is too full. And before I try to resize my disks I need my data off.

Btw gparted shows my 1st nvme drive in a weird state.

I expect no response and I did not ask for help so maximum respect non the less.

But in case anyone has some then maximum will get a new definition.
 
I just foolhardischly and somewhat romantically had hoped that ZFS just deals with stuff like this.. But I will admit I am wrong
 
ok I changed my mind.. I am now in the house of ZFS. then I might better ask for help while I have the chance.

How does one work around this gparted funky business?

1697996712639.png

I feel a bit weird barking up an freebsd tree while this mayhem was caused by an fork :(

nah i am stickiing to my guns gals and gents..;) Do not we all need a official one liner (ok ok , 3 or 4 maybe is the max?). to get an exact copy (but ids) while attaching?
 
You must log in or register to reply here.
Back
Top