Setting ZFS for Internet Services

[wisdown]

Hey guys,

I am starting to play with FreeBSD for host websites, database and send mail, on first try I have used on FreeBSD in test environment:

http://www.zpanelcp.com/

By the way, on Linux I'm using:

http://www.ispconfig.org/page/home.html

ISPConfig allow me have multiple dedicated servers (example: 1 Linux server for email, 1 for web, 1 for database, etc...) but I don't know how to do it on Zpanel.

On Linux I have read a lot about permissions, how to mount dedicated partitions for /var/www, /var/db, /var/logs, /var/mail, etc...

My question is about what's should be an good setup making the datasets on ZFS?
Including the permissions too

I read the online documentation from oracle website about the parameters, and then started read the book:

FreeBSD Mastery: ZFS
Michael W Lucas

And then, for my surprise I have discovered the public ZFS are not same of Oracle...

By the way the book in my opinion is all about how ZFS works, there no real and usable hints in real world...

Would you guys share some links about hints, share your setup, etc...?

After archive same result from Linux, I will migrate all servers to FreeBSD, so main focus is:

Set ZFS for be an web-host reseller (because this I need focus in security) doing the things on hard way, in another words, by hand since I want to replicate having one server per service and didn't find an good panel for help me on it.
(Initial services: Bind + Apache + PureFTPD + Postfix + Courier + Dovecot + MySQL)

PS: I have read about how bad is use thin disks with ZFS, but on the book the author said about create the datasets with -s avoid many issues... Just saying about it because I'm using ESXi as Host.

Thanks in advance!

 

[wisdown]

I'm thinking about changing the question...
Do you guys use servers running ZFS in a production environment like hosting websites?

Asking because my searches over internet almost stuck on same point:

ZFS is for file server like FreeNAS and others...

Haven't found tutorials or example about use it with Apache for example to be an web-host...

 

[Jeckt]

I've never messed with setting up ZFS for system directories past what the default installer does automatically. The main question comes down to how much RAM do you have, and how much do you want the ARC cache to take up. Depending on how much you want to allocate to ZFS, you should probably set a hard value in /boot/loader.conf using vfs.zfs.arc_max .

I'm not aware of anything special required for the services you've listed with the exception of MySQL. You will want to create a separate filesystem for wherever you put the MySQL data. For this filesystem zfs set recordsize=16k <filesystem>, so that ZFS and MySQL agree on how much data to pull off the disk when records are accessed. Set ZFS caching to metadata only for this directory ( zfs set primarycache=metadata <filesystem>). ZFS caches things in memory however MySQL is already doing this. Also skip the InnoDB double write buffer (zfs makes this unneccesary): in my.cnf

innodb_doublewrite = no

I read most of those based off of a MySQL optimization page (also seems to apply to PostgreSQL), but can't recall where it was. It's just in my pile of notes now.

 

[wisdown]

First let me say, I am novice on FreeBSD, and still learning, so sorry for noob questions.

Then we ready to continue, let me elaborate using my scenario and then elaborate more the question:

My current setup is:

2 DNS Server (Bind) Running FreeBSD 10.1
1 Web Server (Apache) Running Debian
1 Mail Server (Default to ISPConfig if I remember right Dovecot and Postfix) Running Debian
1 Databse Server (MySQL) Running Debian

I already have did some tests trying run FreeBSD on my old laptop, and unfortunately the graphics performance was bad because Nvidia driver. This old notebook works nice with Nouveau. Because of this I'm on Ubuntu (Nouveau working nice here). On those tests I have learned how to install FreeBSD without the installer, making the slices by hand and extracting the image to slices, I like learning it!

If I understood right, on ZFS the sub-folders inheritance permissions from top-folders, we can change it applying special permissions on specific folder, but for me at least, this make more hard to manage (and remember) all those specifics things...

On Linux world for example I do an special slice on LVM for /var/www, set the desired permissions for each subfolder (clients), enable quota and then I'm ready to go.

Reading about the ZFS on FreeBSD the things get a bit more complicated, for example, for now I will not be able to use snapshots from ZFS anymore, because on my read from the book FreeBSD Mastery, seems the snapshots spend the quota limit from clients. What would make a lot of problems to me...

From this exact point I am stuck guessing ways to avoid this kinda of problem, my first guess was around make an dataset for /var/www and then do an soft link on zroot dataset, should be an secure way?

Other problem is around jails, reading the manual seems ZFS make an specific dataset for Jails, so should I don't care about customize datasets on top system since zfs jails will not inheritance the structure made?

And on install make an specific dataset for jails using zfs jail option?

About Hardware:

I am using Virtual System managed by VMware ESXi free Version, is around 16 GB RAM (using 4GB for every server - ESXi reports max usage 2 GB RAM per virtual machine and there no SWAP / Baloon / Compression on memory) and 4 HDs low performance (7200 RPM 64 MB cache), the goal is host a few low traffic sites (around 50 if possible).
In future I'm thinking about change from HD to SSHD for virtual machines and add 2 SSDs for Cache and SWAP managed by ESXi, inside of virtual Machines I don't set Cache and Log partitions for ZFS (maybe in future after get better hardware)