ZFS Boot issues when creating raidz zpool storage.

ruchir15

New Member


Messages: 18

I have 3 disk attached to the system and I wanted to setup it as raidz, after running set commands by gpart show output looks like below (vtpd0 , da0 and da1 are disk attached to system )

Code:
root@mfsbsd:/temp # gpart show
=>       40  167772080  vtbd0  GPT  (80G)
         40       1024      1  freebsd-boot  (512K)
       1064        984         - free -  (492K)
       2048    4194304      2  freebsd-swap  (2.0G)
    4196352  163573760      3  freebsd-zfs  (78G)

167770112       2008         - free -  (1.0M)=>       40  167772080  da0  GPT  (80G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352  163573760    3  freebsd-zfs  (78G)

167770112       2008       - free -  (1.0M)=>       40  167772080  da1  GPT  (80G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352  163573760    3  freebsd-zfs  (78G)
  167770112       2008       - free -  (1.0M)


then I create the zpool raidz setup with below command


Code:
zpool create -f -o altroot=/mnt datapool raidz vtbd0p3 da0p3 da1p3

root@mfsbsd:~ # zpool status
  pool: datapool
state: ONLINE
  scan: none requested
config:    NAME         STATE     READ WRITE CKSUM
    datapool     ONLINE       0     0     0
      raidz1-0   ONLINE       0     0     0
        vtbd0p3  ONLINE       0     0     0
        da0p3    ONLINE       0     0     0
        da1p3    ONLINE       0     0     0


I setup bootfs record as below
Code:
zfs create -o mountpoint=/ -o canmount=noauto datapool/boot
zpool set bootfs=datapool/boot datapool

reboot the machine and it fails with blew error


Code:
BOOTING FROM HARD Disk….

gptzfsboot error 1 lba 1030176
gptzfsboot error 1 lba 1030688
gptzfsboot error 1 lba 1031176

ZFS:  I/o error - all block copies unavailable
ZFS : I/o error - all block copies unavailable
Invalid format

Can’t find /boot/loader

Can’t find /boot/kernel/kernel

FreeBsd/x86 boot
Default datapool/boot:/boot/kernel/kernel
Boot:
|
Can’t find /boot/kernel/kernel



The same configuration works well if it is a single disk configuration and I setup it without raidZ (with single disk) .i.e.
zpool create -f -o altroot=/mnt datapool vtbd0p3

Kindly suggest what Option I could try to fix this
 
OP
R

ruchir15

New Member


Messages: 18

We are mainly facing issue around boot-ablity based on the Environment where we do multiple disk configuration.
basically at boot time if loader is able to see all configured disk in environment system boots up fine else issue around bootablity.

To overcome this issue we did zpool configuration like below


Code:
root@mfsbsd:/bootpool # gpart show
=>       40  167772080  vtbd0  GPT  (80G)
         40       1024      1  freebsd-boot  (512K)
       1064        984         - free -  (492K)
       2048    4194304      2  freebsd-swap  (2.0G)
    4196352    4194304      3  freebsd-zfs  (2.0G)
    8390656  159379456      4  freebsd-zfs  (76G)
  167770112       2008         - free -  (1.0M)
=>       40  167772080  da1  GPT  (80G)
         40       2008       - free -  (1.0M)
       2048    4194304    1  freebsd-swap  (2.0G)
    4196352  163573760    2  freebsd-zfs  (78G)
  167770112       2008       - free -  (1.0M)
=>       40  167772080  da0  GPT  (80G)
         40       1024    1  freebsd-boot  (512K)
       1064        984       - free -  (492K)
       2048    4194304    2  freebsd-swap  (2.0G)
    4196352    4194304    3  freebsd-zfs  (2.0G)
    8390656  159379456    4  freebsd-zfs  (76G)
  167770112       2008       - free -  (1.0M)
root@mfsbsd:/bootpool # zpool status
  pool: bootpool
 state: ONLINE
  scan: none requested
config:
    NAME                STATE     READ WRITE CKSUM
    bootpool            ONLINE       0     0     0
      mirror-0          ONLINE       0     0     0
        gpt/disk0-boot  ONLINE       0     0     0
        gpt/disk1-boot  ONLINE       0     0     0
errors: No known data errors
  pool: datapool
 state: ONLINE
  scan: none requested
config:
    NAME           STATE     READ WRITE CKSUM
    datapool       ONLINE       0     0     0
      raidz1-0     ONLINE       0     0     0
        gpt/disk0  ONLINE       0     0     0
        gpt/disk1  ONLINE       0     0     0
        gpt/disk2  ONLINE       0     0     0
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 11,982
Messages: 38,438

Don't put two pools on a single disk. You're also mixing virtual (vtbd0) disks with physical disks (ad0,ad1). While that might work it's not a recommended configuration.
 
OP
R

ruchir15

New Member


Messages: 18

SirDice : This setup is on a digital ocean droplet. where vtbd0 is primary disk and da0 and da1 SSD volumes attached to droplet.

I dont want to waste the space on primary disk that is why I am creating 4 partition on it.

I am using part3 on vtbd0 and da0 (volume1) for my bootpool

and rest of space for raidzi .i.e. part4 of vtbd0, part4 of da0 and all space for da1 for my raidz configuration.

is it recommend ? or you have any other suggestion for me
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 11,982
Messages: 38,438

or you have any other suggestion for me
Use a smaller disk for vtbd0, then you're not "wasting" anything. Or mirror it with two smaller virtual disks if you insist on having a mirror. When you mix different type of disks (virtual and physical SSD) your performance is going to be dictated by the slowest one (typically the virtual disk). So your performance is going to suck and you're now wasting performance for the sake of a bit of space. Performance is expensive, space is cheap.
 
Top