ZFS on Root questions

X

xy16644

I am considering using ZFS on my root partition. Currently my machine has two 80GB SATA hard drives (mirrored) and 1GB of RAM. From what I understand ZFS needs lots of memory...1 or 2GB RAM per 1TB of data.

Is it practical for me to even consider ZFS on root considering my type of configuration?

If I was to go ahead of this I would be wiping the existing drives/partitions, creating ZFS on root and restoring from a dump file (which will have FreeBSD 8.2 RELEASE) and using ZFS mirroring.

I assume that if I start using ZFS on root that I cannot use dump anymore to backup my entire system to a single dump file (for disaster recovery). Is this where ZFS send/receive comes into it?

Also, considering the size of my disks, would it still work to use GPT instead of MBR?

Appreciate any comments or input!

PS: Should I go to ZFS on root and restore my system successfully to it, I would be upgrading from 8.2 to 9.0 RELEASE shortly after. Is this where snapshots can help me rollback if something goes wrong?
 
Perhaps the first question to answer is "Why do you want to use ZFS?" Unless ZFS provides something that you are going to use, UFS in its various flavors is pretty well proven, reliable, and can do the mirroring you're asking for, probably with fewer system resources.

"Practical to consider" -- yes, practical for the kind of use you want to give it, TBD

GPT is pretty much independent of disk size. Unless you need compatibility with an OS that can't deal with GPT, there is little reason, in my opinion, to use MBR.
 
From the specs I'll go ahead and assume that this is your personal PC and not a server or commercial box.
Is it practical for me to even consider ZFS on root considering my type of configuration?
Click to expand...
Yes, as long as the system is 64 and not 32. 32 just won't work with low memory.
I would be upgrading from 8.2 to 9.0 RELEASE shortly after.
Click to expand...
A must. ZFS v28 is what you should use.
Is this where snapshots can help me rollback if something goes wrong?
Click to expand...
Exactly. You can take maual snapshots, automate the process, manage the snapshots and a lot more. ZFS send/receive is for backups over the network.
would it still work to use GPT instead of MBR?
Click to expand...
GPT all the way. Also, in FreeBSD partition/slice disk structure, ZFS + MBR = virtual file system inside a virtual partition (ada0s1a as oposed to ada0p1).
From what I understand ZFS needs lots of memory
Click to expand...
The memory requirement is a function of datasets. For example, ZFS recommends an individual dataset for each user (zpool/users/zaphod for example). You can see how memory requirement would go up if the box were a diskless-client server with 50 or 100 users. For your box, it's a trivial issue as you will have 5-10 datasets.

My 2 cents:
My hardware when I first started with FreeBSD had 1G RAM. As a desktop machine I had no problems at all. Granted, I made a custom kernel so as to trim down to max the kernel footprint, but that's just me and I would have done it even if the HDD was UFS. Several tips for starting out:
- Read zfs(8)() and zpool(8)(). ZFS has so many features it will make you giddy.
- Read Instalation for GPT HDD: http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot
- With RAM < 2, ZFS pre-fetch will be disabled (not a problem for a desktop). You can do a lot of tuning for ZFS (I did not do any because of time shortage).
- Create only 1 zpool. I created 3, for my own reasons and have observed that accesing data on multiple zpools slow the HDD down compared to if I only had 1 zpool.
- I had some problems with HDD timeouts and could not figure out if it was ZFS. The problem went away with a new board. I suspect the issue was more the crappy SATA cables. Described here.
- Once you have it set up, you can read about ZIL and L2ARC.

@jef: ZFS has so many features, buttons, gadgets that once you go ZFS you really would not consider any other FS in my view. Where to start on giving the reasons? Automatic error check and correct, reporting, no more need for HDD partitioning for likes of /var /home etc, on-the-fly compression for specified datasets, encryption, speed. It's just leaps and bounds ahead. Plus learning and practicing with it now will keep you ahead of the curve.
 
You guys have given me much to think about and for that I thank you!

@jef: Theres many reasons I want to use ZFS: Snapshots, checksums, error checking and many others I can't think of :e Also, I don't dual boot, this is a dedicated FreeBSD server.

@Beeblebrox: You are correct, this is a PC and not a proper server. Its an HP dc7100 (no laughing please!). Yes its nothing special hardware wise but I have been running it non stop for 3 years with no issues (touch wood and all that). I host all my stuff on it and a friends domain for email. Its runs fantastically. The only complaint? Its slow to rebuild all my ports and to upgrade from source. No big deal but worth mentioning.

I'm afraid I do run the i386 version of FreeBSD on my machine so that may mean no ZFS for me? If I want to go to the amd64 version I assume thats a rebuild? (ie: no upgrade path from i386 to amd64?).

I really want ZFS for the snapshots mainly. Then when I want to upgrade the OS and/or ports I can take a snapshot beforehand so if something goes wrong I can rollback. Also, snapshots are fast to take...to make a dump of the system to file takes almost an hour.

GPT sounds like the way to go then. I always thought GPT was *only* for large disks.

It sounds like I need two things: More RAM and an amd64 install of FreeBSD 9?
 
It sounds like I need two things: More RAM and an amd64 install of FreeBSD 9?
Click to expand...
No, and unfortunately NO.
- You do not need more RAM, ZFS will work with 1G
- To install amd64, you need 64-bit hardware (new motherboard new CPU). Can't convert a 32-Bit system to 64-Bit (unless you have Jedi skills).
I do not know the RAM requirements for 32-bit systems, but I would not spend my money on such RAM unless it was almost free.

You can find some older and good amd64 hardware on-line. Don't go to hardware shops. Look on e-bay etc. Research the motherboards, preferably with on-board GPU and look for cheap deals. 32-bit is dead and if you start looking now, after some time you will learn how to spot deals for hardware. One of my favorites: http://www.cpubenchmark.net/
 
Thanks again Beeblebrox.

Maybe I should reveal some of my plans to build a new (cutting edge) server in the next 6 months. :e

I want to build a new server with FreeBSD 9.0 RELEASE amd64 on some really cool hardware. I'll be using it to host all of my stuff (web, database, monitoring, email, SSH etc), my friends domain (email) and would like to use it as a file server for my media server.

The specs will be:

Xeon E5-2620 2Ghz CPU (with builtin AES instructions)
16GB ECC Ram (maybe even 32GB...we'll see)
SuperMicro X9SRE-F motherboard
2 x 128GB SSD drives
4 x 3TB SATA 7200RPM drives

The two SSD drives will be mirrored for the root partition with ZFS and be used for OS and port related storage.

The SATA drives will be used to store my media center video. I was thinking of have one ZFS storage pool consisting of two vdevs...each mirrored. So 2 x 3TB in one vdev and another 2 x 3TB in another vdev. This is striping over two mirrors? I assume I can grow the pool later on by adding another mirrored vdev? This should give me about 6TB of unformatted disk space.

Hopefully this will happen later on in the year! But until that time I think I will continue using UFS on my current server.

I chose the above mentioned Xeon as it has the AES instructions built in. I will be using geli to encrypt everything and want to do that in hardware.

The new server with ZFS should be an interesting project! :e:e
 
@xy16644,

Just a friendly advice. Don't waste the SSD just for the OS. 8GB are more than adequate for the core OS plus all the software you need on a server. You can use the remaining space for SWAP + L2ARC caching.

If you need to cache ports or packages, you can always mount them in your DATA pool.

Regards
 
A fine answer as to "Why?" -- Sometimes the goal doesn't justify the path.

If you're not in a production environment, you can run ZFS just fine on a 32-bit system with 1 GB of RAM. Behind me is an Atom 330 doing just that with the ZFS drive on USB and it hums along fine with a dozen jails running a variety of lightly-loaded services. I do use lzjb compression on just about everything and it and haven't had any panics. I just slapped on a USB drive and started migrating things over bit by bit to see how it played out and what kind of management I wanted to do. I've got a few dozen ZFS filesystems on that machine. I've also got two other Atom 330 / 1 GB machines running full ZFS, mirrored, and most filesystems with lzjb compression.

Moving to FreeBSD 9 is sort of a "must" in my book -- ZFS is more stable and mature. It took me a lazy day to get an FreeBSD 9 box up and running and things transferred over.

Even with what you have listed as your target goals, you don't need a lot of processing power. I agree with gkontos that the SSDs are a waste the way you are proposing using them. The slowest thing in a FreeBSD 9 boot feels like the splash screen "wait for options" delay and your streamed media isn't going to need to come off the drives any faster than a couple megabits per second -- save yourself some money!

The SSDs would be nice to be able to be mounted for /usr/src and /usr/obj or the like if you do a lot of builds and have a fast enough CPU that you get disk I/O limited. Even that can be handled by the 64GB drives.

I think that is is important that your OS is separate from your "data" pools, especially if they become complex. I'm nearly at the point of just using a pair of 8 GB microSDs in USB readers for the OS at the whopping cost of around $25 for the pair. I just have too many other things on my plate and enough old drives lying around to use for the OS.
 
Many thanks for all the input. This is giving me lots to think about which is what I want :e

I spent some time reading up about ZFS ZIL and L2ARC yesterday. The summary I took away from it all was:

ZIL (can) improve write performance and needs a mirrored SSD

L2ARC (can) improve read performance and can be used with a single SSD

So it got me thinking about a few things:

1) How big should my swap partition be with 16 or 32GB RAM

2) Is it worth me even considering using ZIL and/or L2ARC?

3) Can I use ZIL and L2ARC on the same mirrored SSD drives where my root partition is?

4) Considering I'll be using a striped mirror pool, can using an SSD for ZIL/L2ARC be a bottleneck?

5) Speaking of streaming media. How much bandwidth does one need to stream 1080p BluRay ISOs?

6) The other thing I have been thinking about is, how should I present the storage on the server to the media center server? iSCSI? SMB/CIFS?

One thing I forget to mention was, theres a small chance I may want to run some VMs on this machine with VirtualBox (no GUI so headless).

Currently my entire server uses just 17GB of space and this is from 3 years of use and MANY ports installed. I don't use it to store any media though.

In terms of backing up the ZFS partitions, would it make sense to have a single drive in the machine that you can send ZFS snapshots to? Or would it be better to store this on another machine?

Lots to think about and thanks for all your help/input! :stud
 
You must log in or register to reply here.
Back
Top