RAIDz1,2,3 not booting

da1 · Aug 31, 2011

Hi guys,

I'm trying to make a bootable ZFS raidz2, 8.2 amd64 box with 4 drives in it. Following the wiki all goes well but after I reboot, I'm presented with:

Code:

ZFS: i/o error - all blocks copies unavailable
ZFS: can't read MOS object directory
Can't find root filesystem - giving up
ZFS: unexpected object set type 0
ZFS: unexpected object set type 0

FreeBSD/x86 boot
Default: zroot:/boot/kernel/kernel
boot:
ZFS: unexpected object set type 0

FreeBSD/x86 boot
Default: zroot:/boot/kernel/kernel
boot:

If I do "status" I get:

Code:

FreeBSD/x86 boot
Default: zroot:/boot/kernel/kernel
boot: status   pool:zroot
config
        NAME STATE
        zroot ONLINE
         raidz2 ONLINE
           /dev/gpt/da0 ONLINE
           /dev/gpt/da1 ONLINE
           /dev/gpt/da2 ONLINE
           /dev/gpt/da3 ONLINE


FreeBSD/x86 boot
Default: zroot:/boot/kernel/kernel
boot:

so clearly, the pool exists, is visible and the disks are ok. I booted into FIXIT and checked rc.conf, loader.conf and fstab and everyting is ok.

I tried in VirtualBox too but I get the same behavior. Tomorrow I will try again, in another machine but I would like to hear some opinions about this.

BTW: I use the same commands for a mirror setup and a 1 disk setup and it boots fine.

Ideas ?

jem · Aug 31, 2011

Can you paste the output of
# gpart show

da1 · Aug 31, 2011

Sure.

Code:

=>   34  16777149   da0   GPT   (8.0G)
     34       128     1   freebsd-boot (64K)
    162   2097152     2   freebsd-swap (1.0G)
2097314  14679869     3   freebsd-zfs (7.0G)

=>   34  16777149   da1   GPT   (8.0G)
     34       128     1   freebsd-boot (64K)
    162   2097152     2   freebsd-swap (1.0G)
2097314  14679869     3   freebsd-zfs (7.0G)

=>   34  16777149   da2   GPT   (8.0G)
     34       128     1   freebsd-boot (64K)
    162   2097152     2   freebsd-swap (1.0G)
2097314  14679869     3   freebsd-zfs (7.0G)

=>   34  16777149   da3   GPT   (8.0G)
     34       128     1   freebsd-boot (64K)
    162   2097152     2   freebsd-swap (1.0G)
2097314  14679869     3   freebsd-zfs (7.0G)

Crest · Sep 1, 2011

Last time i checked gptzfsboot only supported pools with one mirrored or raidz1 vdev.

da1 · Sep 1, 2011

Same situation for raidz1, raidz2, raidz3.

graudeejs · Sep 1, 2011

Can you show us /boot/loader.conf please?

da1 · Sep 1, 2011

loader.conf

Code:

ahci_load="YES"
atapicam_load="YES"

#ZFS kernel modules
zfs_load="YES"
vfs.root.mountfrom="zfs:zroot"

# ZFS tunnables
## for AHCI
vfs.zfs.vdev.min_pending=4              #default=4
vfs.zfs.vdev.max_pending=8              #default = 35
## NO AHCI
#vfs.zfs.vdev.min_pending=4             #default=4
#vfs.zfs.vdev.max_pending=8             #default = 35
 
# Increase vm.kmem_size to allow for ZFS ARC to utilise more memory.
# vm.kmem_size="2048M"                  # Not needed in 8.2
# vm.kmem_size_max="2048M"              # Not needed in 8.2
vfs.zfs.arc_max="2048M"

# Disable ZFS prefetching (we will not disable it because we have 6GB of RAM)
# http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html
# vfs.zfs.prefetch_disable="0"

# Decrease ZFS txg timeout value from 30 (default) to 5 seconds.  This
# should increase throughput and decrease the \"bursty\" stalls that
# happen during immense I/O with ZFS.
# http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html
# http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html
vfs.zfs.txg.timeout="5"

# Target seconds to sync a txg
vfs.zfs.txg.synctime="1"

And since I'm at it, rc.conf

Code:

zfs_enable="YES"
sshd_enable="YES"

and /etc/fstab

Code:

/dev/gpt/swap none swap sw 0 0

graudeejs · Sep 1, 2011

looks fine...
You probably did this, but check that you have:
# zpool set bootfs=zroot zroot
It's a common mistake I forgot to do

da1 · Sep 1, 2011

Checks out fine. I remember I forgot to issue that cmd once too .... lesson learned, never again

.

graudeejs · Sep 1, 2011

oh, another one... did you copy /boot/zfs/zpool.cache over to your pool?

da1 · Sep 1, 2011

I did:

Code:

cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

da1 · Sep 1, 2011

A colleague of mine told me that as far as he knows, on FreeBSD it isn't possible to boot from raidz*. I have personally never setup a box with that config but since there are tutorials on the wiki, I would expect it to work (makes no sense to write something on the wiki if it isn't working, right ?).

graudeejs · Sep 1, 2011

I boot from raidz (raidz1)

da1 · Sep 1, 2011

Ok, can you give me some specs (hdd's, etc) and the script you used to set it up (or link if no script) ? Maybe I did some typo somewhere and I cannot see it.

graudeejs · Sep 1, 2011

Unfortunately I did it all by hand....
With the help of FreeBSD wiki (However I actually remember all the process off my memory)

3x HDDs: http://www.wdc.com/en/products/products.aspx?id=110

da1 · Sep 2, 2011

UPDATE: I tried a raidz(1) on a normal machine (with 3 disks) and it works. It seems that VIrtualBox (4.0.12) is having some issues here.

Quis · Dec 25, 2011

I've got the same problem. I can mount it when booting from a non-ZFS disk but not when booting itself.

8.2 amd64, raidz1 (3x 2TB)

I did do some tricks to make sure ZFS uses the right blocksize (Use gnop to fake the sector-size at 4096 instead of 512 (which the disk reports)) (Advanced Drive Format crap). Might be related?