GEOM explanation

I

ioverreact

Could anyone here point me to a good tutorial on GEOM?

I've read the handbook section and a few of Michael Lucas' books (Absolute FreeBSD and the shorter pocket sized one on filesystems) but still can't get my head around what it is or where it fits in.

I've found his books genuiniely very helpful but the chapters on GEOM are really confusing.

I didn't need to understand it in order to install FreeBSD as the handbook explains how to complete an install step by step.

I would like to learn more about how the system works though.
 
If you understand the concept of "stacking" as in the typical OSI network stack, you have a good start on GEOM.
Here's my understanding which may or may not reflect "reality".

Start at physical device, like a spinning disk.
System goes through the boot, detects the hardware, says "I need a device driver"
Device Driver loads, emits an event "I have found hardware and this is what I've found".
Now GEOM starts to get involved. Keep thinking "provider" and "consumer". Device Driver is providing something.
Now "GEOM Manager" (not the real name, but conceptually something that understands things and hooks up providers and consumers) peeks at the provider (GEOM calls this tasting). Finds some metadata, say "this is part of a gmirror" and starts to create the gmirror (loading modules, etc).
A Device Driver provides a different disk device, GEOM tastes it, realizes "Oh that's the other part of the gmirror I found", GEOM Manager links the two and exposes /dev/gmirror/MyMirror.
Then things like fstab can reference /dev/gmirror/MyMirror.

So GEOM is basically "stackable transforms".
 
mer said:
If you understand the concept of "stacking" as in the typical OSI network stack, you have a good start on GEOM.
Here's my understanding which may or may not reflect "reality".

Start at physical device, like a spinning disk.
System goes through the boot, detects the hardware, says "I need a device driver"
Device Driver loads, emits an event "I have found hardware and this is what I've found".
Now GEOM starts to get involved. Keep thinking "provider" and "consumer". Device Driver is providing something.
Now "GEOM Manager" (not the real name, but conceptually something that understands things and hooks up providers and consumers) peeks at the provider (GEOM calls this tasting). Finds some metadata, say "this is part of a gmirror" and starts to create the gmirror (loading modules, etc).
A Device Driver provides a different disk device, GEOM tastes it, realizes "Oh that's the other part of the gmirror I found", GEOM Manager links the two and exposes /dev/gmirror/MyMirror.
Then things like fstab can reference /dev/gmirror/MyMirror.

So GEOM is basically "stackable transforms".
Click to expand...
Ahhh I think I might have got it. Correct me if I'm barking up the wrong tree though.

In terms of it being like a stack i.e. the network stack or tcp/ip layers, it sits somewhere between device drivers and the user applications?

If that's the case, does it work like LVM does in Linux? For example where you can have multiple disks, each with a number of partitions, and various file systems set up but all are looked after / managed by LVM, which the user can use as a tool for adding more disks etc...?

And is it baked into a particular file system i.e. UFS? or a part of the operating system? or simply an add on tool? I only ask because, I've managed to install the whole operating system including a working desktop without using it (that I'm aware of).

I'll apologise in advance if any of this is old hat / obvious. It's just I've read and re-read the chapters and am unsure of what it is.
 
ioverreact said:
In terms of it being like a stack i.e. the network stack or tcp/ip layers, it sits somewhere between device drivers and the user applications?
Click to expand...
No, GEOM is more of a framework. Each layer in the stack has a specific function. Like the OSI model has 7 layers, a GEOM "model" can have multiple layers, all stacked on top of the previous. GEOM just makes it so each layer can be stacked on top of another, in, mostly, any order you like.
 
SirDice said:
No, GEOM is more of a framework. Each layer in the stack has a specific function. Like the OSI model has 7 layers, a GEOM "model" can have multiple layers, all stacked on top of the previous. GEOM just makes it so each layer can be stacked on top of another, in, mostly, any order you like.
Click to expand...
I think I understand how it's used because the Lucas books and the man pages give some good examples i.e. if I want a RAID system with encryption I can stack the modules in the order I want them i.e. set up the RAID then do the encryption rather than be forced to do it the other way round. So modules can be used at will in a very flexible order.

The part I'm struggling to understand is where does GEOM sit and what is it a part of. I may have even misunderstood nodes and where they sit.

Would it be right in this order:

- physical disk
- driver
- node
- GEOM
- user?

And is GEOM a part of FreeBSD or is it tied to a specific file system i.e I would need to choose UFS over ZFS to use it? or vice versa?

I've just had another read of the chapter in "FreeBSD Mastery Storage Essentials" and I'm still a bit confused.

Thanks for the help btw :)
 
ioverreact said:
And is GEOM a part of FreeBSD or is it tied to a specific file system i.e I would need to choose UFS over ZFS to use it? or vice versa?
Click to expand...
FreeBSD, basically "before" any filesystems in the stack. Nominally right after the device driver. (This may not be 100% technically accurate but is reasonably close functionally).

Physical devices normally present device nodes like /dev/ada0, /dev/ada0p1....
GEOM presents device nodes like /dev/gmirror/MyMirror (which would consist of /dev/ada0, /dev/ada1)
Device nodes need to have filesystems on top of them, so by creating a UFS filesystem (say /usr/home) on top of /dev/gmirror/MyMirror you are creating a "mirrored UFS filesystem", transparent to a human user. Writes to /usr/home get distributed to the devices making up the mirror, you don't need to do anything.

Example:
MSDOS FS or FAT32 does not support mirroring. But you can take 2 physical devices, create a GEOM mirror (gmirror) then create a FAT32 filesystem on top of the mirror.
That means you wind up with a mirrored FAT32 filesystem.
Swap devices/partitions. Create a gmirror and then swapon the gmirror device for a mirrored swap.

RAID other than mirror. There are GEOM classes to create things like RADI3, RAID5 etc so you create the underlying GEOM RAID then put UFS filesystems on top of that. Gives you a UFS filesystem that has redundancy.

Your RAID/Encryption:
create GEOM RAID, encrypt "the RAID" instead of individual devices, create the single filesystem on top of the encryption.
GEOMs stack: so in theory (not practical example) you could create a bunch of GEOM mirrors then create a GEOM RAID on top of those mirrors giving you a "RAID of Mirrors". Create filesystems on top of that and you get a bunch of redundancy.
 
ioverreact said:
The part I'm struggling to understand is where does GEOM sit and what is it a part of. I may have even misunderstood nodes and where they sit.
Click to expand...

GEOM is a kernel level framework. It's a part of the kernel.

ioverreact said:
And is GEOM a part of FreeBSD or is it tied to a specific file system i.e I would need to choose UFS over ZFS to use it? or vice versa?
Click to expand...

It's part of FreeBSD but it is independent of any filesystem it sits below. I believe is sits right on top of the CAM Layer, and below UFS/ZFS. For example, GELI and gpart are both GEOM classes used to encrypt and partition providers (ie. a disk, SSD, or thumb drive). They can be used with either UFS or ZFS. It's a complete framework for manipulating disks.
 
ioverreact said:
Could anyone here point me to a good tutorial on GEOM?
[...]
I would like to learn more about how the system works though.
Click to expand...
If, after reading Michael Lucas' two books, you still have a feeling like "but the chapters on GEOM are really confusing", I guess you didn't (yet) have formed a satisfactory model/image of what GEOM is or is supposed to be; perhaps lacking detail or another point of view.

Besides geom(4), there's ample (reference) documentation and tutorial info:
  1. Gunion(8): a new GEOM utility in the FreeBSD Kernel By: Marshall Kirk McKusick - video presentation
    - Gunion(8): a new GEOM utility in the FreeBSD Kernel - BSDCAN 2023 - description & slides
    - extended introduction: general GEOM info and its place inside the OS;
    - specific gunion(8) at the second part.
  2. Frequently Asked Questions about GEOM - 2004
    - including entry points to Presentations and slides/tutorials by Poul-Henning Kamp
  3. Poul-Henning Kamp - Publication LIST
    - entry points to GEOM tutorials and presentations during BSDCon 2003 and BSDCAN 2004
    - including a paper and slides about GEOM Based Disk Encryption
  4. Michael W Lucas - FreeBSD Mastery: Specialty Filesystems
    I presume you are referring to this book by "the shorter pocket sized one on filesystems", but see also Storage Essentials:
  5. Michael W Lucas - FreeBSD Mastery: Storage Essentials
    - much more (detailed) info on GEOM than #4
  6. FreeBSD kernel GEOM code Reference Manual - 2007
    This goes beyond any mere tutorial, but maybe helpful when looking at the current GEOM source code. I have no idea if much has changed since 2007 and therefore how precisely this Reference Manual relates to the present day FreeBSD code base.
 
Erichans said:
If, after reading Michael Lucas' two books, you still have a feeling like "but the chapters on GEOM are really confusing", I guess you didn't (yet) have formed a satisfactory model/image of what GEOM is or is supposed to be; perhaps lacking detail or another point of view.

Besides geom(4), there's ample (reference) documentation and tutorial info:
  1. Gunion(8): a new GEOM utility in the FreeBSD Kernel By: Marshall Kirk McKusick - video presentation
    - Gunion(8): a new GEOM utility in the FreeBSD Kernel - BSDCAN 2023 - description & slides
    - extended introduction: general GEOM info and its place inside the OS;
    - specific gunion(8) at the second part.
  2. Frequently Asked Questions about GEOM - 2004
    - including entry points to Presentations and slides/tutorials by Poul-Henning Kamp
  3. Poul-Henning Kamp - Publication LIST
    - entry points to GEOM tutorials and presentations during BSDCon 2003 and BSDCAN 2004
    - including a paper and slides about GEOM Based Disk Encryption
  4. Michael W Lucas - FreeBSD Mastery: Specialty Filesystems
    I presume you are referring to this book by "the shorter pocket sized one on filesystems", but see also Storage Essentials:
  5. Michael W Lucas - FreeBSD Mastery: Storage Essentials
    - much more (detailed) info on GEOM than #4
  6. FreeBSD kernel GEOM code Reference Manual - 2007
    This goes beyond any mere tutorial, but maybe helpful when looking at the current GEOM source code. I have no idea if much has changed since 2007 and therefore how precisely this Reference Manual relates to the present day FreeBSD code base.
Click to expand...
I've been reading Absolute FreeBSD and number 5. on your list, the storage essentials.

Can I just check after re-reading the various chapters that I've understood it correctly?

So far it looks like a toolbox (aka framework) of macros that make managing disks easier. It uses OO language to describe its features and is flexible in how its applied.

So similar to an OO language, you can create an instance of a class i.e. a mirror, which you can use alongside other instances i.e. another RAID type or encryption, in any order you choose. And that you perform these tasks when setting up a new disk for a fresh install.

If I can give an example of where I think it would be used and how it would work can someone correct me if I get it wrong?

Say I have a desktop with two hard disks. I install FreeBSD using a live usb. During the install I decide to set up the disks manually. I'm assuming I will be able to use various instances/geoms to 'transform the disks' i.e. manually partition the drives, set up RAID and encryption and some sort of LVM. I then install the file system of my choice i.e. ZFS on the logical volume I've just created. Then, rather than show physical device nodes, the new system will show a single volume that I can later add to with more physical drives should I wish.

I imagine it can get a lot more complex if you already have the operating system installed on one disk but wish to add another and then set up RAID etc...but that's something I'll look at later once I've had a chance to practice the basics.

Thanks for all the help and extra sources, I'll peruse them over the weekend.

I've also got second hand copies of FreeBSD in 24 hours and FreeBSD unleashed. They're much easier to read for a non professional (I'm not a sys admin) but I've noticed a lot of the info is outdated so I need to be careful i.e. commands like pkg_add and use of cvsup. I've been double checking everything with the hand book and touch wood all is going well so far ??
 
Don't get lost in the nitty gritty details (unless you want to). GEOM is a unique architectural feat, but with ZFS available you probably can get along with a lot less GEOM knowledge nowadays.

Before ZFS came to FreeBSD, when you wanted a RAID-mirror solution, you basically had two options: hardware RAID or a software GEOM based solution. With ZFS you have another choice (with it a lot more and better overall functionality), bringing its own redundancy, data protection and "speed" for that. You can use ZFS on top of a GEOM mirror, and that works, but it is less than an all-in-one package as compared to using ZFS for that. ZFS "likes" to be as close to the hardware as it can.

Something similar holds for encryption, but I'm less familiar with that.

Note that, because of its place in the filesystem stack, even when using ZFS for a ZFS-mirror or ZFS RAIDZn solution, ZFS lives on top of GEOM; see The Kernel I/O Structure diagram at the beginning of Kirk McKusick's presentation, taken from The Design and Implementation of the FreeBSD Operating System, 2nd Edition—the GEOM layer is "less thick" (when ZFS handles the mirror functionality) as it packs less functionality.

Practically speaking, one of the things of GEOM that may be important, especially when using ZFS, is some knowledge about (GEOM) disk withering. By the way, there's a whole world of FreeBSD besides GEOM for you to explore :)
 
Beastie7 said:
GEOM is a kernel level framework. It's a part of the kernel.



It's part of FreeBSD but it is independent of any filesystem it sits below. I believe is sits right on top of the CAM Layer, and below UFS/ZFS. For example, GELI and gpart are both GEOM classes used to encrypt and partition providers (ie. a disk, SSD, or thumb drive). They can be used with either UFS or ZFS. It's a complete framework for manipulating disks.
Click to expand...
GELI + ZFS need a bit more care, it is very easy to broke MBR/GPT table with it. At least in my exp.
 
You must log in or register to reply here.
Back
Top