NAS-in-a-VM for ZFS-everywhere

RJVB

RJVB

Hi,

That's a bit of a vague title for a question that's clear enough. I hope I'm posting it in the most appropriate forum.

I've come to like ZFS a lot over the last few years. Hardware and software considerations keep me from running FreeBSD or PC-BSD as my main OS (that's still OS X for now) but I do use ZFS when and where I can.

I didn't for that new spacey external drive I took into service, because I wanted to be able to use it with MS Windows too. AFAIK there still is no (good) solution to use ZFS pools directly on MS Windows (correct?).

My question: how feasible would it be to set up a minimal FreeBSD system for use in a VM using something like VirtualBox providing direct access to a disk, and then to export the mounted datasets to the host? How large would such a VM be (on disk and in RAM) and would that indeed provide faster I/O than you get with a typical NAS?
And: am I reinventing wheels again? :)
 
On the FreeNAS forums, they are pretty insistent that running it as a VM is a bad idea.

Despite that, the desire is understandable. In your case, the VM acts as a relatively large ZFS driver.

Disk space for the VM operating system would be minimal. RAM would be the trick. Some people claim that ZFS with only 1G is possible, but it probably won't be much fun. 2G maybe, 4G probably. RAM is cheap now, so if the machine will take it, add memory and dedicate it to the VM.

Performance... don't know. Depends on how fast transfers are between the VM and host. And there will be interactions, because the VM shares the host CPU and other subsystems.

Personally, I'd rather have a separate machine with its own CPU and RAM.
 
Exactly, the VM would be a large driver, probably more so than "relative" suggests, and with a fixed memory footprint.

I've already run Kubuntu from a ZoL pool in a VM with 4Gb or RAM, meaning a complete KDE4 desktop in which I may even have done a kernel build. So yeah, 4Gb should be enough.

I suppose that on a reasonably fast, modern host, communication between guest and host should be a lot faster than disk I/O over the network. It might even use DMA if what I've been told about "localhost" protocols is true.
 
RJVB said:
It might even use DMA if what I've been told about "localhost" protocols is true.
Click to expand...

DMA does not apply to the application level at all, it is strictly a kernel driver level technique to use an external "CPU" to transfer data over PCI/PCIe (the older ISA bus does DMA as well) bus so that the main CPU does not have to be involved in the transfer.
 
I think the point of my source (former coworker) was that the TCP/IP stack can use DMA when communications are on the same host. He also used the actual words (in French, at that) rather than the acronym, so the whole hardware and involvement of the CPU (or not) aspects probably didn't come into play at all. "Memory to memory" might be a better translation.

And it concerned MS Windows (so "kernel driver level" might not be a very relevant concept ;))
 
You could set up FreeNAS or FreeBSD in a virtual machine and run it with ZFS. It'll work, performance may not be the best since you're using a VM and transferring data through the VM and over a network connection. So there are a couple of bottlenecks that will reduce performance.

That being said, either FreeBSD or FreeNAS will work as a file server in a VM with 1GB or 2GB of RAM. So long as you're not running other services, that much RAM will be enough for the OS + ZFS. Personally, I'd probably use FreeNAS for this, since it'll pretty much automate setting up network shares for you.
 
I run such a setup combined with ProxMox. So I got a 3-node-cluster with proxmox and in the "biggest" server I have 2 HBAs passed through to a FreeNAS-VM. No performance-issues at all - 16 GB RAM, 2 HBAs and 4 Cores combined with striped mirrors (4 x 2 HD) just works excellent.

Regards
Markus
 
Good, thanks. I'll know what to tinker with once I have a real reason to start using ZFS disks directly under MS Windows.
If indeed FreeNAS will automatically share new mounts that are added without intervention that'd be very useful; there's already the hurdle that you'd need to ensure that a newly connected external disk is seen by the VM guest and not by the host.

As a side-ways related question: what's the official position on using ZFS on disks connected over USB? The ZFSonOSX devs have reasons to advise against it. I cannot recall if that is the case with ZoL too (I certainly never had issues with it). Excluding USB leaves little options to use ZFS on an external non-networked driver for the majority of consumer hardware (even Macs, nowadays)...
 
Last week one of the controllers in my server decided to die. As I didn't have a replacement on short notice I put three disks (one RAID-Z pool) into a USB enclosure. The enclosure shows all three disks separately. ZFS picked up the drives (that were previously attached to a controller) without problems. It's horribly slow (as everything has to be pushed over a single USB 2.0 connection) but it works without a hitch.
 
Be careful with the enclosures. Some cheaper ones have bugs in the firmware that cause FreeBSD to detect the disks with wrong geometry, usually 4 or 8 sectors less than what's really on the disk.
 
Yeah, I think I have a "good" enclosure, it was an expensive one. It has USB and eSATA connections. The pool isn't really important, it had a couple of my test jails on it and a bunch of poudriere build trees. It would be a bit of a setback if I lost it but there's nothing I can't rebuild from scratch.

My new controller card (LSI 9702-8i) will hopefully arrive in the next couple days.
 
You must log in or register to reply here.
Back
Top