Planning a ZFS build

[Again]

Hello, I'm a 98% *nix noob, but since I saw Vista I knew that where Windows goes is not the place I want to be..A quarter ago I decided to try to leave it with the upcoming PC update. It came earlier than I expected, and though I'm really scared, I'm gonna try it. The decision is to use PC-BSD because it has ZFS and I expect that setting up a workable environment with it is gonna be easier than with regular FreeBSD...
My configuration is:

• 4 hard drives, each one different, 230 GB - 1.3 TB.
• 1 HDD is WD EARS.
• I want ZFS with copies=2 to have almost-RAID1 with good utilization of what I already have.
• I want gzip -9 on OS (as it's mostly static except for /var and /tmp, right?) and LZJB in other places.

Questions:

1. How does ZFS recover when a HDD dies? Do I have to insert another disk or can I force it to just add another copy to one of the alive ones? If the 230 died soon I'd still have enough space to not have to buy a replacement for a while and I'm wondering if it wouldn't be better to not use it at all.
2. Should I use just 1 ZFS pool or would it be better to slice the disks? I'd rather avoid slicing because I have no idea how much space do I need for OS and somehow it always ends up with having one slice full before the other fills in half...and I end up moving stuff between them. Blah, I hate making such mess. But I'm unsure how will I handle possible situations like a switch to linux when everything is in 1 pool.
3. FUSE on linux supports only ZFS pool v13 and I'd rather have it as an option. I didn't read changelogs to see what am I missing yet but I don't think it's a killer. Can I create such pool with the latest version of FreeBSD or do I have to use some older one?
4. How about ZFS stability in BSD? Reading various related stuff I saw like 2 cases when sb. lost a pool. My backup disk is already 20% too small to keep all my stuff, so I'm gonna loose some music. Nothing really terrible, but ripping it again is quite a lot of work.
5. WD EARS. I've seen like 5 threads on various forums discussing problems with it. Generally the talk is about it's troubles in RAIDZ, but some say just about ZFS and I'm confused, is it going to cause problems with regular striping?
6. Do I want to have OS deduplicated? I guess it's going to save little, but with practically no cost.
7. In order to improve compression I'd like to increase block size. Is it a good idea? If yes, at what point can I expect a notable performance drop? I'm talking about regular desktop use; net, programming, gaming, multimedia.
8. How to set it up? I reckon that the best way is to use a PC-BSD Live DVD to prepare a pool and then somehow figure out how to install the OS over it. I saw that there are some FreeBSD guides, but the fact that PC-BSD has a different installer makes me wonder whether they apply. Will I be able to do it? Please remember that though rather determined, I'm just a noob...
9. Anything you think I might be missing?

 

[graudeejs]

* I want gzip -9 on OS (as it's mostly static except for /var and /tmp, right?) and LZJB in other places.

AFAIK root, or at least /boot can't be compressed. If you compress /boot you won't be able to boot (unless you boot from flash)

2) Should I use just 1 ZFS pool or would it be better to slice the disks? I'd rather avoid slicing because I have no idea how much space do I need for OS and somehow it always ends up with having one slice full before the other fills in half...and I end up moving stuff between them. Blah, I hate making such mess. But I'm unsure how will I handle possible situations like a switch to linux when everything is in 1 pool.

If you want to boot from HDD, then you will need to make slices.
1x freebsd-boot and 1x for freebsd-zfs gpt schemes.
I'd also suggest adding 1x freebsd-swap scheme

3) How about ZFS stability in BSD? Reading various related stuff I saw like 2 cases when sb. lost a pool. My backup disk is already 20% too small to keep all my stuff, so I'm gonna loose some music. Nothing really terrible, but ripping it again is quite a lot of work.

For me zfs is rock-stable on laptop, dekstop pc and server

4) WD EARS. I've seen like 5 threads on various forums discussing problems with it. Generally the talk is about it's troubles in RAIDZ, but some say just about ZFS and I'm confused, is it going to cause problems with regular striping?

Relay on zraid if possible, it's FAST

5) Do I want to have OS deduplicated? I guess it's going to save little, but with practically no cost.

Dunno, AFAIK, if you use FreeBSD -RELEASE then deduption is not available (yet)

6) In order to improve compression I'd like to increase block size. Is it a good idea? If yes, at what point can I expect a notable performance drop? I'm talking about regular desktop use; net, programming, gaming, multimedia.

Music, video archives are already compressed, you won't gain anything.
Compression will introduce little latency.

7) How to set it up? I reckon that the best way is to use a PC-BSD Live DVD to prepare a pool and then somehow figure out how to install the OS over it. I saw that there are some FreeBSD guides, but the fact that PC-BSD has a different installer makes me wonder whether they apply. Will I be able to do it? Please remember that though rather determined, I'm just a noob...

I mix knowledge of this thread http://www.daemonforums.org/showthread.php?t=1538
With some howto from http://wiki.freebsd.org/ZFS (depending how you want to setup your PC)
Generally it's easy

8) Anything you think I might be missing?

Read: http://forums.freebsd.org/showthread.php?t=6230

NOTE: everything I said is based on my own experience

 

[dandelion]

3) FUSE on linux supports only ZFS pool v13 and I'd rather have it as an option. I didn't read changelogs to see what am I missing yet but I don't think it's a killer. Can I create such pool with the latest version of FreeBSD or do I have to use some older one?

zpool(8) supports creating pools with lower version numbers (-o version=...), e.g. for grub2/linux/netbsd compatability. However, such pools wouldn't support features like user/group quota (v15) or deduplication (v21).

AFAIK root, or at least /boot can't be compressed. If you compress /boot you won't be able to boot (unless you boot from flash)

Only gzip compression is not supported (PR kern/153173) by (gpt)zfsboot. Booting from lzjb or zle compressed pools should and does work for me.

 

[graudeejs]

zpool(8) supports creating pools with lower version numbers (-o version=...), e.g. for grub2/linux/netbsd compatability. However, such pools wouldn't support features like user/group quota (v15) or deduplication (v21).Only gzip compression is not supported (PR kern/153173) by (gpt)zfsboot. Booting from lzjb or zle compressed pools should and does work for me.

Wow, didn't know that, Thanks

 

[Again]

Thank you both for the answers.

If you want to boot from HDD, then you will need to make slices.
1x freebsd-boot and 1x for freebsd-zfs gpt schemes.
I'd also suggest adding 1x freebsd-swap scheme

How does it relate to http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition ?

I searched and found that the boot partition is something really small, so the problems are so much smaller anyway.
I wonder whether ZFS will disable caching though...is it a case with BSD too?

Relay on zraid if possible, it's FAST

You mean RAIDZ? Since all my drives are very different and I don't want to buy new ones it's not really an option. If I really wanted I could do this with tiny slices, but it's more hassle than it's worth. It's very possible that I'll get similar drives and do it at some point though.

Dunno, AFAIK, if you use FreeBSD -RELEASE then deduption is not available (yet)

Indeed, I remembered wrong. It's not a problem though, until dedupe can be done offline I have little use for it anyway.

Music, video archives are already compressed, you won't gain anything.

Yep. But I have enough of compressible data to make it worthwhile.

Compression will introduce little latency.

Yes, but is it really something to worry about? I haven't seen any benchmarks, so all I can do is estimate. Pessimistically:
50 MB/s gzip decompression, 8 KB block size = 0.16 ms. There is likely some static overhead, let's double it. 0.32 ms.
HDD seek takes about 9 ms. With SSDs it may be something. Possibly with 128k blocks too. But unless I miss something big, in most cases it shouldn't be even measurable.
Based on this I'm gonna try 8k for most of the filesystem. The data set benchmarked is not really representable for me, but I haven't seen anything else and now I got more important things than benchmarks to do.
For some things like data/usr/src I can't imagine use case that wouldn't involve accessing whole files, so even 128k block size shouldn't hurt. I don't think it's worth bothering though...but I'm a compression junkie and I think I'll do it anyway.

I mix knowledge of this thread http://www.daemonforums.org/showthread.php?t=1538
With some howto from http://wiki.freebsd.org/ZFS (depending how you want to setup your PC)
Generally it's easy

Thanks for the link, hope it works with PC-BSD too...

Read: http://forums.freebsd.org/showthread.php?t=6230

NOTE: everything I said is based on my own experience

I have it bookmarked already.

zpool(8) supports creating pools with lower version numbers (-o version=...), e.g. for grub2/linux/netbsd compatability. However, such pools wouldn't support features like user/group quota (v15) or deduplication (v21).

Good, I don't really need either feature so I'll get compatibility painlessly. Thanks.

Only gzip compression is not supported (PR kern/153173) by (gpt)zfsboot. Booting from lzjb or zle compressed pools should and does work for me.

What part of the filesystem does it apply to? /boot?

 

[dandelion]

What part of the filesystem does it apply to? /boot?

(gpt)zfsboot (stage2) and zfsloader (stage3) from boot(8). Note NULLs for gzip in zio_decompress_func_t in zio_compress_table in sys/cddl/boot/zfs/zfssubr.c which likely means it's not implemented.

So, yeah, only dataset containing /boot where zfsloader/grub2 and kernel reside cannot currently be compressed by gzip.