ZFS zfs: get sizes in megabytes

[crypt47]

Hello, I struggle to find an option which sets custom units to list/show datasets and zvol properties. Anybody knows?

 

[yuripv79]

There does not seem to be any, see (specifically) this comment: https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/lib/libzutil/zutil_nicenum.c#n113.

 

[crypt47]

There does not seem to be any, see (specifically) this comment: https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/lib/libzutil/zutil_nicenum.c#n113.

I see.. well... i'm kinda have to meditate on that... you see, I used linux LVM for a long time and used to clone volumes (block devs) over net... for that you have to know the exact size... otherwise it doesn't fit on disk... so... when I have an LVM vol it provides the size... then I can create ZVOL with that particular size... and now I don't have that data anymore on FreeBSD except for "zfs history" command... What if I want to block copy ZVOL -> ZVOL?

probably have to use fdisk or something like that... but! it gives a bit smaller size, then what was in zfs history.

 

[yuripv79]

zfs get/set/list and zpool get/set (likey other subcommands as well) all have the -p option to display raw values, could be useful here.

 

[sko]

I see.. well... i'm kinda have to meditate on that... you see, I used linux LVM for a long time and used to clone volumes (block devs) over net... for that you have to know the exact size... otherwise it doesn't fit on disk... so... when I have an LVM vol it provides the size... then I can create ZVOL with that particular size... and now I don't have that data anymore on FreeBSD except for "zfs history" command... What if I want to block copy ZVOL -> ZVOL?

probably have to use fdisk or something like that... but! it gives a bit smaller size, then what was in zfs history.

That's not how size and physical space allocation with zfs works... If you dump a dataset/zvol to a file its size may vary depending on compression and amount of metadata that dataset brings along (which also varies depending on e.g. stripe-/blocksize). By far the worst relation between 'physical' and actual dataset/zvol size would be data on raidz pools - the huge amount of padding and parity data introduces a significant error in size "on zfs" and "physical" size if you were to dump a dataset/zvol to a file; let alone the actually moved data by zfs send (which involves a lot of additional metadata, e.g. for snapshots)

 

[VladiBG]

zpool get -p -H size | awk 'size=$3/1024^2 {print $1,$2,size "MB"}'

 

[crypt47]

zfs get/set/list and zpool get/set (likey other subcommands as well) all have the -p option to display raw values, could be useful here.

Oh! Fine! I didn't get right 'Display numbers in parsable (exact) values.' Only saw 'parsable' not the exact. Thanks a lot!

 

[crypt47]

That's not how size and physical space allocation with zfs works... If you dump a dataset/zvol to a file its size may vary depending on compression and amount of metadata that dataset brings along (which also varies depending on e.g. stripe-/blocksize). By far the worst relation between 'physical' and actual dataset/zvol size would be data on raidz pools - the huge amount of padding and parity data introduces a significant error in size "on zfs" and "physical" size if you were to dump a dataset/zvol to a file; let alone the actually moved data by zfs send (which involves a lot of additional metadata, e.g. for snapshots)

zfs send/recv doesn't look attractive as it doesn't send deltas, i believe.

 

[crypt47]

zpool get -p -H size | awk 'size=$3/1024^2 {print $1,$2,size "MB"}'

No, I think I need volsize property, not the size that you've used. volsize/1024/1024 matches the size I've used with LVM. It solves the problem of the topic.

 

[sko]

I haven't tried it yet as I did with LVM. Suppose I want two ZVOLs to send data from one to another, how should I allocate the second? I have to do some test first...

You could just clone the zvol if it is OK that the clones will depend on the original (snapshot). For e.g. VMs that are cloned from a master image this is the preferred way and saves a lot of physical space.
If you actually *need* completely independent copies, just send|recv to get a full copy of the whole zvol without any dependencies or other strings attached between the two. but regarding physical space this is horribly inefficient...

LVM is horrible - especially their snapshots are practically useless for somewhat fast changing volumes (we had VMs where LVM snapshots wouldn't finish within 24h and the temp volume for the deltas filled up all available space). DON'T transfer any concept from LVM to zfs - that won't work, because zfs does everything much more efficient and elegant.

So regarding your zvols: you don't have to specify any size because you don't want to copy them on a block-level e.g. via dd - just use the built-in capabilities like clone or send|recv and zfs will handle all the space allocation for you 'the right way'™

zfs send/recv doesn't look attractive as it doesn't send deltas. Am I right?

Of course you can send incrementals. That's the main point of snapshots which are always used for zfs send|recv, otherwise you wouldn't be able to work with a dataset while it is sent, or you would need some (bad) workaround like a temporary buffer for all deltas that accumulate while the transfer is running (-> that's what LVM does).

 

[crypt47]

You could just ...

> You could just clone the zvol if it is OK that the clones will depend on the original (snapshot).

We are speaking about remote machine here, so no clonning, only sending snapshots.

> So regarding your zvols: you don't have to specify any size because you don't want to copy them on a block-level

What if it doesn't fit?

> Of course you can send incrementals. That's the main point of snapshots which are always used for zfs send|recv

Hmmm.. Let's say we have to identical zvols about 300Gb. One has changed. We want to send the diff to another. So you're saying... we can send 1Gb delta? How will it look like?

I also want to transfer the encrypted zvols btw.

 

[Eric A. Borisch]

zfs send/recv doesn't look attractive as it doesn't send deltas, i believe.

How do you mean? When you send the incremental update between from snapshot fs@a to fs@b, it certainly only sends the new changes (deltas), not the whole fs@b. (@a needs to exist at the recv location for this to work, clearly.)

 

[sko]

What if it doesn't fit?

you know the size and reservation of the zvol. also you should always leave enough headroom on a zpool for housekeeping, metadata etc - rough rule of thumb is at ~80% allocated space you should extend the pool.
So if you have to calculate on a megabyte-scale if the data might fit, it won't fit or it will bring the pool to a crawl.

Hmmm.. Let's say we have to identical zvols about 300Gb. One has changed. We want to send the diff to another. So you're saying... we can send 1Gb delta? How it will be look like?

Take a new snapshot and send the delta between the last synced and new snapshot...
zfs-send(8)

Make sure to read (at least) the topics on zfs snapshots and replication in the handbook:

Chapter 22. The Z File System (ZFS)

ZFS is an advanced file system designed to solve major problems found in previous storage subsystem software

docs.freebsd.org

Chapter 22. The Z File System (ZFS)

ZFS is an advanced file system designed to solve major problems found in previous storage subsystem software

docs.freebsd.org

With ZFS you can take snapshots in very short intervals and send them to other hosts e.g. for backup purposes. Our replications usually run on a hourly basis, with snapshots being taken anywhere from dailies to 5-minute intervals, depending on the data or VM in that dataset/volume.
Also you don't have to make any assumptions or calculations about the size of those datasets - if they are transferred between pools (of the same vdev structure - e.g. mirrors), their size as shown by zfs list remain roughly the same. The reservation of a vdev is always taken into account and included in the "used" space - on backup pools you might want to unset the "reservation" and "refreservation" values to save space.


Again: zfs has fundamentally different concepts than simple volume managers like LVM. Please read at least the handbook chapter on ZFS and mentioned manpages to avoid pitfalls due to e.g. wrong assumptions about some features and so you don't make your life harder than it needs to be - ZFS simplifies A LOT of work if you use it right.

 

[crypt47]

How do you mean? When you send the incremental update between from snapshot fs@a to fs@b, it certainly only sends the new changes (deltas), not the whole fs@b. (@a needs to exist at the recv location for this to work, clearly.)

I'm new here. May assume wrong.

 

[crypt47]

you know the size ..

> you know the size and reservation of the zvol.

yes, and this contradicts your previous statement.

> you don't have to specify any size because you don't want to copy them on a block-level

I need to know the size and that's why I've started the topic.

> Please read at least the handbook chapter on ZFS

Only general stuff in there. I still need to google for if it's possible to send encrypted zvol anyway.

 

[Eric A. Borisch]

To get all properties (with sizes in bytes) of a zvol:

zfs get -p all zpool/path/to/zvol

You are looking for 'volsize' (you could replace 'all' with 'volsize' to only get this value); it is the logical size of the zvol volume (if you made a "40G" zvol, this will be 40G, even with nothing written to it) in bytes. I believe this is what you are looking for. (See zfsprops(7) for descriptions of other parameters.)

That said, if you want to duplicate the zvol, a snapshot followed by send/recv would be the preferred 'zfs' way to do it, or even perhaps a clone (writable new volume with reference back to the original volume for unchanged blocks) depending on what you want, but you can always just make a new zvol and then use dd or other userland tools to copy block-by-block.

Some useful initial background to review / reference: zfsconcepts(7)

 

[astyle]

Only general stuff in there. I still need to google for if it's possible to send encrypted zvol anyway

Something wrong with zfs send? My educated guess is that it won't care, just have zfs receive ready on the other end

 

[Eric A. Borisch]

Only general stuff in there. I still need to google for if it's possible to send encrypted zvol anyway.

Assuming you're talking zfs encryption (used with zfs-load-key), and not some other layer, or encryption of the communication pipe, zfs send -w ... will send the encrypted stream (won't be decryptable on the backup system unless it has access to the keys.)

See zfs-send(8) for more details and discussion.

 

[chungy]

By default, ZFS volumes are created with a refreservation pre-calculated to account for metadata overhead (including raidz[123] overhead); this is the same as setting refreservation=auto. The only time you might actually care about checking space before-hand is if you're using refreservation=none, but if you're combining these two things, you've designed your system in a very wrong way.

As long as you stick to the default refreservation (which is sensible), ZFS won't allow you to fill your volume more than it possibly can, other datasets won't write so much data that the volume could not be filled, and you cannot receive a volume onto a new pool where it wouldn't fit. Setting "refreservation=none" can violate all of these principles, and that's why you should avoid doing that.

 

[crypt47]

Something wrong with zfs send? My educated guess is that it won't care, just have zfs receive ready on the other end

I just don't know yet. Haven't a chance to try it before.

Assuming you're talking zfs encryption (used with zfs-load-key), and not some other layer, or encryption of the communication pipe, zfs send -w ... will send the encrypted stream (won't be decryptable on the backup system unless it has access to the keys.)

See zfs-send(8) for more details and discussion.

And zfs-load-key sеrves both volume and it's snapshots at the same time while keys are loaded into memory? I think this will work just fine, thank you.

 

[chungy]

And zfs-load-key sеrves both volume and it's snapshots at the same time while keys are loaded into memory? I think this will work just fine, thank you.

Snapshots always have the same encryptionroot as their parent dataset, this cannot be changed (snapshots are fundamentally pointers to old views of data: none of the data is copied; it can't be anything other than the same encryptionroot).

If you send raw sendstreams, the receiving hot doesn't need the key at all. Besides being unable to mount the dataset, you can still receive incrementals, check for damage, and do a raw send from that host to anywhere else. A non-raw send requires the sending host to have the keys loaded, and it will be sent unencrypted (the receiving host may encrypt it with a new key at that point).

Read zfsconcepts(7), zfsprops(7), and zfs-send(8)

 

[astyle]

By default, ZFS volumes are created with a refreservation pre-calculated to account for metadata overhead (including raidz[123] overhead); this is the same as setting refreservation=auto. The only time you might actually care about checking space before-hand is if you're using refreservation=none, but if you're combining these two things, you've designed your system in a very wrong way.

As long as you stick to the default refreservation (which is sensible), ZFS won't allow you to fill your volume more than it possibly can, other datasets won't write so much data that the volume could not be filled, and you cannot receive a volume onto a new pool where it wouldn't fit. Setting "refreservation=none" can violate all of these principles, and that's why you should avoid doing that.

I didn't know ZFS is that friggin' smart!