Gstripe array gone after reboot, very strange behaviour.

[gnoma]

Hello,

I use gstripe with four disks of 1 TB each.

I create my array with gstripe, make the GPT table, store data on it and after system reboots I got no array. The four disks are there, but gstripe status says there's a gstripe device. Recreating the device, makes it empty, no GPT table and no filesystem.

I tried to create the array and reboot instantly, did not create the table and did not touch anything in the array. Just create it and reboot. Still no array after reboot.

FreeBSD hostname.lan 9.1-RELEASE FreeBSD 9.1-RELEASE #0: Tue Jul  2 10:34:07 EEST 2013     root@storage1.deltanews.lan:/usr/obj/usr/src/sys/PFKERNEL  amd64

File /boot/loader.conf

zfs_load="YES"
vfs.root.mountfrom="zfs:zroot"
geom_stripe_load="YES"

File /etc/sysctl.conf

#security.bsd.see_other_uids=0
kern.geom.stripe.fast=1
net.inet.carp.preempt=1
net.inet.carp.allow=1
net.inet.carp.log=1
net.inet.carp.drop_echoed=1

Before reboot everything was fine, this is after reboot:

root@storage1:/ # uptime
 3:30PM  up 12 mins, 1 user, load averages: 0.01, 0.24, 0.25
root@storage1:/ # kldstat
Id Refs Address            Size     Name
 1   14 0xffffffff80200000 1335a60  kernel
 2    1 0xffffffff81536000 2084f8   zfs.ko
 3    2 0xffffffff8173f000 5c68     opensolaris.ko
 4    1 0xffffffff81745000 9668     geom_stripe.ko
 5    1 0xffffffff81812000 2579     geom_gate.ko
root@storage1:/ # gstripe status
root@storage1:/ # 
root@storage1:/ # gstripe list
root@storage1:/ #

To create the gstripe array, I use this:

root@storage1:/ # gstripe create -s 32k -v stripe4 /dev/ada0 /dev/ada1 /dev/ada3 /dev/ada5
Done.
root@storage1:/ # gstripe status
          Name  Status  Components
stripe/stripe4      UP  ada0
                        ada1
                        ada3
                        ada5
root@storage1:/ #

And the logs:

root@Storage1:/var/log # grep -ir stripe *
messages:Jul  2 15:15:55 storage1 kernel: GEOM_STRIPE: Device stripe4 created (id=3294592924).
messages:Jul  2 15:15:55 storage1 kernel: GEOM_STRIPE: Disk ada0 attached to stripe4.
messages:Jul  2 15:15:55 storage1 kernel: GEOM_STRIPE: Disk ada1 attached to stripe4.
messages:Jul  2 15:15:55 storage1 kernel: GEOM_STRIPE: Disk ada3 attached to stripe4.
messages:Jul  2 15:15:55 storage1 kernel: GEOM_STRIPE: Disk ada5 attached to stripe4.
messages:Jul  2 15:15:55 storage1 kernel: GEOM_STRIPE: Device stripe4 activated.

I have worked with gmirror and graid5 and had no such problems at all. Anybody have idea why this is happening? Google didn't help me a lot, because I got no errors, and even no informational events in the logs.

Thank you.

 

[wblock@]

There is a conflict between the backup GPT table and the gstripe(8) metadata. Both want to store things at the end of the disk.Whichever one is written last overwrites the one that was there before.

One solution is using MBR instead of GPT, but MBR is limited to 2 TB partitions.

Another solution is ZFS, which I think can do RAID0.

 

[kpa]

ZFS does striping (RAID0) by default if you create a pool without specifying redundancy, for example zpool create mypool ada0 ada1 would create a striped RAID0 pool.

 

[gnoma]

Hello,

I should really pay more attention to the man pages. I am sorry for the stupid post, it's all described in the man page of gstripe http://www.unix.com/man-page/FreeBSD/8/gstripe/. The solution was to use gstripe label, not gstripe create. This is just normal behavior.

To make the topic not so pointless I'll mention that I am using GEOM instead of ZFS, because the purpose of the array is iSCSI target which is running best on block devices and HAST, which works only on block devices. I am using it for VMFS filesystems, on top of them are running virtual machines on FreeBSD and Solaris, both on ZFS.

Still I need high performance and I can not get it from ZFS without SSD disks to store the ZIL. It's all described here http://forums.freebsd.org/showthread.php?t=40179. ZFS is a great file system and volume management, of course the first thing I did building this solution was to try it on ZFS, but it just doesn't serve best this purpose.

And even so, with

gstripe

I still got serious performance issue, I got performance of a single disk instead of four disks. It appears that there is problem with some disk controllers, described here http://lists.freebsd.org/pipermail/freebsd-geom/2009-January/003291.html.

So adding "-s 32k" solved it completely. This is the final looks of it gstripe label -s 32k -v stripe4 /dev/ada0 /dev/ada1 /dev/ada3 /dev/ada5.

Thank you.