question about devices

W

wonslung

I have a rocketraid 2340 and right now i have 8 out of 16 drives connected

they are da0-7 right now.

my questions is this:
If i use them for ZFS and add more drives later do i run any risk of the device names changing on me? i know in linux it would happen from time to time so i just want to be sure if i should do anything special.

Also, with zfs i hear it's better to use entire devices, what i'm not clear on is whether or not i'm supposed to use the device name like ad0 or if i'm supposed to make a freebsd partition first and maybe one big slice.

thanks for the help.
 
Yes, it's possible for the device names to change, based on the order that they are connected to the controller. For example, if a drive dies, you shutdown the box, pull the drive, and boot with a missing drive, all the devices after that one will be renamed (da4 becomes da3, for example). Or, if you boot with a USB stick connected, the kernel could set that as da0 and cause all the rest of the devices to be renumbered. This causes ZFS to implode. :) Running with this setup for too long can corrupt the pool (we learnt this the hard way).

If you shutdown right away, and connect a drive back into the same spot, and reboot, things will usually start to work correctly again.

However, there is a very simple fix for this: use glabel(8) to label your drives. Then use label/<name> instead of da3 when creating vdevs. For example, if you label your disks "disk0", "disk1", and "disk2" (for da0 da1 and da2), then you can use # zpool create pool raidz1 label/disk0 label/disk1 label/disk2. Then, it doesn't matter where on the controller the drive is connected, as the kernel will know to use the label instead of the physical device name to access the drives (and will re-map things internally as needed).

Yes, it's always better to use full drives with ZFS. And no, you don't need to format them, partition them, or do anything with them. ZFS handles all of that automatically.

With FreeBSD, you can use full drives, individual slices, individual partitions, or even files to create ZFS vdevs. Due to the nature of the GEOM framework, performance should not be too different between full/slice/partition. (On Solaris, if ZFS can't use the whole drive, then it disables the onboard write cache, which really kills performance, so they recommend only using full drives.)
 
phoenix said:
However, there is a very simple fix for this: use glabel(8) to label your drives. Then use label/<name> instead of da3 when creating vdevs. For example, if you label your disks "disk0", "disk1", and "disk2" (for da0 da1 and da2), then you can use # zpool create pool raidz1 label/disk0 label/disk1 label/disk2.
Click to expand...

Thanks again, you've been my personal savorior.

remember the stuff that i said i ordered? it all got here last week but then the powersupply failed so i ended up having to wait till today..i should have known better probably, as i never intended to go up to 20 drives when i bought it, i originally though i'd never use more than 6

anyways, i just got freebsd installed to my new compact flash card (your idea) and made the mirror, haven't added the second one yet.



edit:

one more thing, if i had set up zfs on another system already without using glabel is there anyway to repair it AFTER the fact?

edit2:

i read the glabel man page and i'm not really sure i understand. it seems to me it only works for partitions or filesystems. how do you set a label for an entire device?

edit3:
i think i got it, read an old post, guess i should have looked=)
does
Code: 
glabel label won0 /dev/da0
create the label won0 for disk da0?
and it's permanant?
so then i could use label/won0 when i create my vdevs
cool
 
wonslung said:
one more thing, if i had set up zfs on another system already without using glabel is there anyway to repair it AFTER the fact?
Click to expand...

Yes. Just label the drive. Then use the zpool replace command to replace each drive with itself, but using the label name instead of the device name. Something like: # zpool replace pool da4 label/somecoolname

i read the glabel man page and i'm not really sure i understand. it seems to me it only works for partitions or filesystems. how do you set a label for an entire device?
Click to expand...

Yeah, the glabel man page can be a bit confusing, as it covers temporary labels, automatic labels, permanent labels, device labels, filesystem labels, and more. :)

The GEOM class "label" (the glabel kernel module) handles reading all the different types of labels, but the glabel command only creates/removes device labels.

glabel label somecoolname da4 will create a permanent device label called "somecoolname", and store it on device "da4". After that, you can use /dev/label/somecoolname (or just label/somecoolname for short) to reference that disk.
 
phoenix said:
Yes. Just label the drive. Then use the zpool replace command to replace each drive with itself, but using the label name instead of the device name. Something like: # zpool replace pool da4 label/somecoolname
Click to expand...

and this doesn't cause any issues? just label the drive then do zfs replace? even for simple striped zfs pools?

thanks, i'll try that
 
It worked on one server, where we started with using the real physical devices, later replaced with the labels. But, I've only done it on the one server. :)
 
This is a stupid oversight on my part, i probably should have asked this:

Once you have things set up and you decide to add more devices, how to you check which labels are attacked to which devices so you can label the NEW devices correctly...i hope that came out better than i think it did :)

what i mean is, lets say you have da0 da1 da2 da3 and you label them won0 won1 won2 and won3 (accordingly)

now lets say you add 3 new drives, when you reboot, they might be da0 da1 da2 da3 da4 da5
how do you tell which ones are won0 won1 won3 won3 because they might have shifted (which is the reason for labels in the first place)
 
# glabel list

That will list out all the labels that the module knows about, and which disks/filesystems/etc they are attached to.

You'll want to pipe it through more, or redirect it to a file, as there's a lot of info in there.
 
phoenix said:
# glabel list

That will list out all the labels that the module knows about, and which disks/filesystems/etc they are attached to.

You'll want to pipe it through more, or redirect it to a file, as there's a lot of info in there.
Click to expand...


cool deal
 
You must log in or register to reply here.
Back
Top