Need help setting up first NAS + ZFS

tl;dr How to setup and manage FreeBSD NAS for hosting iSCSI disks, VM images and snapshots?

I want to setup a FreeBSD NAS which will be used mostly for hosting ISCSI disk images and maybe also as NFS storage.
I'm somewhat familiar with FreeBSD but have no experience with ZFS. As FreeBSD is shifting to ZoL/ZoF this complicates things maybe even more. There are TrueOS images with the new ZFS implementation. Does anyone know if the FreeBSD-13 iso's also already make use of it? If so, did anyone here test it and can maybe share some experiences?
From what I read on ZFS I can use ZVols for storing the ISCSI images and revert the images back to the original state if required. In my case the images itself, however, are used as host for virtual machines and I'm wondering what would be the best strategy to set this up. Virtualization software would be KVM in most cases and VM images would be in qcow2 format. The VM images should also be golden images themselves so that they can also be reverted to a previous state. It should also be possible to easily compare the old and new state for file changes.
I currently see the following options:

1. I put everything i.e. host system and VM images into the same ZVol. Makes the setup somewhat easier, but comparing the states would be more difficult I guess at least for the VM's. A diff would only show that the qcow2 files changed but you can't look into them without mounting the disk and then mounting the qcow2 images.

2. Put everything in a separate ZVol. So VM as well as host image could be stored separately. On the KVM host the VM images could be attached also via iSCSI or some kind of remote storage. I did never test that but I guess it should work? However, I don't know if if zfs diff can look into the images and compare the different snapshots without mounting the disk first.

3. Use the snapshot feature of KVM itself and put everthing in one or more ZVol. Should work, but when I revert the host disk to some previous state I loose some information about the qcow2 snapshots when using separate ZVols for host and VM.

The FreeBSD system itself would be encrypted, however, I need another large disk for storing data and the actual iSCSI/ VM images. This disk also should be encrypted but additionally should be mountable read-write from a linux system in some kind of dual boot setup. ext3/4 rw are supported since the latest release but I think something like LUKS is not. But since ZFS supports encryption and the versions in both FreeBSD and Linux are the same it should be possible to decrypt and mount the same disk from both FreeBSD and Linux?
 
As FreeBSD is shifting to ZoL/ZoF this complicates things maybe even more.
It doesn't. Why would anything change? Besides that, there's nothing to worry about just yet, it's nowhere near any -RELEASE version.

A Zvol acts as a (block) device, not a filesystem. It works the same as a 'real' disk, like da0 for example, and needs to be partitioned and formatted before you can actually put files on there.
 
Well, I'm mostly worried because I have no experience with ZFS yet and I don't want to start from scratch again. I can't use the native encryption yet and would need to set this up again once it is out. It currently looks like the 0.8 release will be out soon for Linux and I could maybe create a deb package myself for installation. However, I'd like to use FreeBSD but don't know if I build FreeBSD from the latest source myself I could already make use of the new features.
 
Back
Top