ZFS native encryption, resizing, bleaching

geek

geek

atomicbeef said:
Encryption is already handled by geli(8), resizing is already possible (here's a tutorial for example), and something like BleachBit will never be part of ZFS.
Click to expand...

Is shrinking the ZFS pool partition currently supported? I recall that only growing is.

geli(8) only encrypts "block devices", so if I want to have both encrypted an unencrypted ZFS datasets, or multiple datasets with different keys, wouldn't that be more inflexible and require a different ZFS pool volume for each key? Also, if RAID5 is to be used, I believe using ZFS RAID on top of GEOM encrypted volumes would be cryptographically less secure because of the parity, and I'm not sure whether it's a good idea performance- or otherwise to build ZFS on top of GEOM RAID.

The ZFS implementation doesn't need to support bleaching. Instead, maybe some third-party program could implement ZFS bleaching support.
 
  1. A pool can never be shrunk. As far as I know, this is and has always been true in every implementation of ZFS.
  2. Last I heard (a few months ago), nobody was interested in implementing encrypted datasets, because it would require drastically altering and complicating quite a few things at the fileystem and kernel levels. Block device-level encryption is both simpler and more secure than dataset-level encryption.
  3. "Shredding" a file on a CoW filesystem is a heck of a hurdle to vault. You'd practically need to break a fundamental feature (arguably the fundamental feature) of ZFS to really make it work. How do you eliminate all traces of a file when a dozen copies of that file, and two dozen copies of its metadata, existOf course, if you're using encrypted block devices and encrypted swap, it's totally unnecessary.
geek said:
geli(8) only encrypts "block devices", so if I want to have both encrypted an unencrypted ZFS datasets, or multiple datasets with different keys, wouldn't that be more inflexible and require a different ZFS pool volume for each key?
Click to expand...

That's true, but the question arises: how many people would ever really need several different datasets with different keys on a single system? It raises the administrative burden, makes real security more complicated, and doesn't really achieve anything that proper user and group management can't.

geek said:
Also, if RAID5 is to be used, I believe using ZFS RAID on top of GEOM encrypted volumes would be cryptographically less secure because of the parity...
Click to expand...

Parity data is stored on a dedicated block device, not a dataset. So the only way to encrypt that data is at the block device level. And that encryption takes place before the data touches the platter.

geek said:
I'm not sure whether it's a good idea performance- or otherwise to build ZFS on top of GEOM RAID.
Click to expand...

It's a very bad idea, but GELI is not part of GEOM RAID.
 
I believe shrinking and bleaching a ZFS are perfectly possible and I could maybe even implement tools for that myself. I asked in my first message whether the information found at those links is sufficient to create a shrinking/bleaching tool for FreeBSD ZFS. And by bleaching I don't mean shredding an individual file, but wiping all unused data areas on the file system image with meaningless data when the file system is unmounted.

Some ZFS implementations already have native encryption. link
 
geek said:
Some ZFS implementations already have native encryption.
Click to expand...

https://lists.freebsd.org/pipermail/freebsd-fs/2016-May/023181.html

That mailing list thread doesn't completely rule out the possibility in the future, but it makes it clear that just because dataset encryption exists in the Solaris implementation doesn't mean it can easily be done in other operating systems, and that the FreeBSD developers haven't really considered it worth their time and energy.

geek said:
And by bleaching I don't mean shredding an individual file, but wiping all unused data areas on the file system image with meaningless data when the file system is unmounted.
Click to expand...

"Unused data areas" is a nebulous term. ZFS keeps a variable number of copies of both data and metadata on-disk---a number that can change pretty frequently---and deliberately spreads those copies around the disk so physical damage/wear to one area doesn't affect multiple copies. You could very well bring the system down and spend an entire day wiping unused areas, only to have unused areas filled with data as soon as the system comes back up. Then have those areas unused again a few hours later.
 
geek said:
Hi. I would like to have some of these features available on FreeBSD ZFS: native encryption, resizing (both shrinking and growing) of ZFS partitions, and BleachBit-style disk space cleaning.
Click to expand...
+1

Unfortunately Oracle closed source ZFS at least one version "too soon" and that might not be even legally possible (ZFS is encumbered by patents) even if it was possible technically (ZFS required re-implementation of large part of Solaris kernel in FreeBSD). I personally always thought that the better strategy for any BSD was to have a native, open source, bsd licenses file system but the life sometime writes its own endings.
 
Very interesting discussion. I was reading elsewhere that now the majority of commits to the OpenZFS umbrella project are now coming from Linux developers. It would seem then that it is only a matter of time before Linux specific features make OpenZFS more proficient on Linux as opposed to FreeBSD. This would be pure speculation on my part however.
 
As with anything that grew within the linux-ecosystem within the last ~2 years I'd be _VERY_ cautious using it anywhere near production systems... Better lean back and observe for a while - and join the passing of the popcorn if it goes downhill (again..) ;)

If you only need/want to allow users to encrypt their home directories (which is the case for ~90% of all requests regarding ZFS encryption...) just use PEFS. It nicely integrates with e.g. PAM and the whole en-/decryption can even be handled on the remote side, so no unencrypted data is ever stored or cached on the server side. PEFS also doesn't interfere with backups - so the server still can back up even at file-level without the horrible workaround of backing up a whole image of the blockdevice (which may be 95% empty...).
I'm using PEFS on top of ZFS on my Laptop for home directories and for several directories on fileservers at work and home and I never want to miss the flexibility of this solution again.
 
getopt said:
Well as long as the LINUX guys don't have committing permissions on FreeBSD ... :)
Click to expand...

Well, at some point it might/will be commited to OpenZFS, so it depends on if/how it will be ported to the FreeBSD implementation of OpenZFS and how well it is reviewed before being commited...
 
geek said:
I believe shrinking and bleaching a ZFS are perfectly possible and I could maybe even implement tools for that myself.
Click to expand...

Good luck! In order to implement shrinking of pools, removal of vdevs, and altering the make-up of vdevs, you'll need to implement the fabled Block Pointer Rewrite feature ... that has been talked about for over 10 years now. There's been a couple of attempts at doing it, but nothing has been committed into the ZFS tree yet, as it's a very hairy beast of a problem to sort out. It basically changes one of the most fundamental features of ZFS, and touches upon/requires change to pretty much every single feature within ZFS.

By all means, give it a try. If you do manage to crack that nut, you'll be heralded as a hero! We've all been waiting for this feature to be implemented in order to "complete" ZFS. :) I won't be holding my breath waiting for it, but I will certainly salute your attempt at it. :D
 
getopt said:
https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-linux/

Do we need to wake up our ZFS-devels or are they testing already?
Anyway don't they need a push of motivation? Cheer them up!

If ZFS on Linux has native encryption and FreeBSD-ZFS does not, guess where I go ... ;)
Click to expand...

That's not how ZFS development works.

Each port is free to work on new features (it's why there's feature flags instead of version numbers). Once those features are stable, then they get submitted upstream to the OpenZFS repo. Once they are accepted into there, any downstream ports are free to include those features in their port.

IOW, there's no need for FreeBSD and Linux and Illumos and TruOS and other devs to all work on the same feature at the same time. Let one of the groups do the dev work on feature X, while another group does work on feature Y, while another group does work on feature Z. They'll all get upstreamed into OpenZFS at some point, and then filter down to all the other OSes. No sense in duplicating work, when everyone can just build off what the others have created.

Have a look at the zpool-features(7) man page (especially the one for 12-CURRENT). Look at the GUID field for each of the feature flags listed in there. You'll see where the feature originated. There's some from Delphix, some from Joyent, some from Open-ZFS, some from Illumos, etc.
 
https://twitter.com/OpenZFSonLinux/status/897157795698442240 merged on the 14th.

I hope that FreeBSD will be not too far behind.

https://github.com/zfsonlinux/zfs/issues/494#issuecomment-189203472 (2016-02-26) and then the most recent mention of FreeBSD in long ZoL PR 4329 (predecessor to 5769) was https://github.com/zfsonlinux/zfs/pull/4329#issuecomment-269171215 (2016-12-26),where Pawel Jakub Dawidek wrote:

We (at Wheel Systems) will probably work on porting ZFS encryption to FreeBSD, unless someone will beat us to it. For us the most important feature is encrypted zfs send, so we may wait until this is done. Still, there are many active committers working on ZFS in FreeBSD, so I'd expect this is going to happen soon.
Click to expand...
 
Ah, I didn't know the FreeNAS people were putting in such efforts. That's fantastic news.


The FreeBSD 12.0 release cycle is currently expected to begin early 2019.[​

wow, still a whole year out though...
 
If you do not need it badly I guess you can just use ZFS without encryption for now and later turn it ON when available, then do some zfs send | zfs recv to have everything encrypted. :beer:
 
As far I understand about the OpenZFS project, each OS implement the features for its own and the code is shared with OpenZFS Project (what is maintained mostly by former SUN people, IIRC), and then the other OSes implement those features later for they-self if they want to.
 
Yes, I don't see anything wrong with what he said there. In fact in the previous question he actually mentions they haven't linked it to any external encryption frameworks as that would make supporting the many systems OpenZFS runs on more awkward. This guy is working on the feature in Linux, just like FreeBSD devs might work on new features in FreeBSD, and it has always been the plan to upstream it to the main OpenZFS codebase. It's up to FreeBSD to keep up to date with the central OpenZFS code. As this is mainly work inside ZFS (rather than memory/disk io, vfs, etc), it should *hopefully* not be that difficult to port as he says.

It's just a shame that with the lack of manpower in the BSDs we now seem to have ended up last in line for ZFS features/improvements.
 
You must log in or register to reply here.
Back
Top