Why use zfs when ufs will do ?

As mer pointed out in post #7, ZFS saves a LOT of headaches and planning. With UFS (and just about any other file system), you have to do a lot of planning ahead, doing some math, making a firm decision at the start of the installation process, and sticking with it. With ZFS, you can fine-tune a lot of details any time after installation, and update that config to fit the situation at hand. That flexibility, the very fact that such a robust feature is available on FreeBSD, and the fact that it's so well integrated into FreeBSD - that was my tipping point for sticking with FreeBSD and leaving Linux behind. With UFS, if I want to change something later, I can't make the change on a running system. Either I'm stuck with the decisions I made when I was installing FreeBSD, or I'm faced with redoing the whole installation process from ground up. Using ZFS means there's one less reason to reinstall the whole system.

I'm having a desktop FreeBSD 14.2 host with 16GB of ram, and recently using zfs.
My windoze-10 vm guest is needing 8GB of ram. This all worked reliably using ufs.
Now the vm keeps pausing. It seems this failure is caused by zfs using up all the memory.
Is changing back to ufs filesystem the best solution ?
If you can afford it, I'd recommend upgrading the RAM, maybe even max out what your mobo can support. I have a laptop with 16 GB of DDR5 RAM, and VirtualBox was kind of slow on that hardware. 10th gen Intel processors have been reported to be rather slow under FreeBSD, too.
 
Thanks for replies.
Before reverting to ufs, I'll have a go with some zfs tuning as suggested, and hope for positive outcomes.
As for RAM, I'm not rushing to buy outdated DDR3 memory modules for a desktop that is perilously close to its use-by date. (BIOS dated 2014).
So I'm likely stuck with 16GB for a while.
 
As for RAM, I'm not rushing to buy outdated DDR3 memory modules for a desktop that is perilously close to its use-by date. (BIOS dated 2014).
So I'm likely stuck with 16GB for a while.
Just to note that these DDR3 modules are pretty affordable in e-Bay these days.

Also, if you have a free SATA interface, I do advise to add L2ARC cache to the pool in the form of a small SSD drive. This will improve the read speed (maybe not for all use cases, but this is worth to try).
 
Also, if you have a free SATA interface, I do advise to add L2ARC cache to the pool in the form of a small SSD drive. This will improve the read speed (maybe not for all use cases, but this is worth to try).
Yes, I do have spare small SSD drives, and an unused SATA3 interface.
BUT as the whole zfs pool is already on a larger SSD, can L2ARC on SSD make any benefit ?
L2ARC cache and main storage would be operating at same speed, and just adding unnecessary overhead to RAM.
Or am I missing something fundamental here ?
 
Yes, I do have spare small SSD drives, and an unused SATA3 interface.
BUT as the whole zfs pool is already on a larger SSD, can L2ARC on SSD make any benefit ?
L2ARC cache and main storage would be operating at same speed, and just adding unnecessary overhead to RAM.
Or am I missing something fundamental here ?
My opinions, based on what I've learned/read about. Others may have different opinions.

Depending on the workload for the specific zpool, L2ARC may help.
On a root dataset/zpool? Probably not.
On a dataset/zpool that holds data files like audio or video that are primarily READ access? Good chance it will help.
ZFS ARC is in memory, has a max size it can grow to.
Servicing a read request from memory is quicker than servicing from the physical device.
Items/blocks that get read over and over (think multiple users streaming 101 Dalmations for the grandkids) wind up in ARC, if they age out of ARC they fall to the L2ARC device.
Now is servicing from L2ARC device faster than servicing from the original physical device if they are the the same read speed? Maybe, maybe not. I think in general it's more "maybe yes" because the L2ARC device is dedicated for one use, where the zpool/dataset will likely have more than one use.

That's where the package zfs-stats come into play. Use it to figure out your current workload/ARC efficiency. If you have low effiency that means most read requests are going back to the physical device; in that case I think L2ARC would help.
 
Yes, I do have spare small SSD drives, and an unused SATA3 interface.
BUT as the whole zfs pool is already on a larger SSD, can L2ARC on SSD make any benefit ?
L2ARC cache and main storage would be operating at same speed, and just adding unnecessary overhead to RAM.
Or am I missing something fundamental here ?
Sometimes L2ARC has a benefit even when the main drive is SSD. It is hard to measure the exact performance, but just thinking about it - there is an extra interface involved and that may introduce some additional parallelism.

Yes, according to manuals, the L2ARC needs some extra RAM for its indexes. After all it is all about experimenting and also probably depends on the actual hardware.

Personally I have noticed a speed gain in case the main pool is on SSD. Just a wild guess - maybe in some cases the system can read data faster from L2ARC than from the main pool on SSD. You can give it a try and if there is no help, then just to remove the L2ARC drive. This is completely safe.
 
Now is servicing from L2ARC device faster than servicing from the original physical device if they are the the same read speed? Maybe, maybe not. I think in general it's more "maybe yes" because the L2ARC device is dedicated for one use, where the zpool/dataset will likely have more than one use.
Agree that this is may-be and real answers may come out of actual experimentation. Also, SSD-s are not equal.
 
  • Like
Reactions: mer
Subjective opinion based on reviews in many threads dedicated to ZFS.
1. You need more than one or two disks for redundancy and creating an array for unforeseen situations.
2. The disks should be able to rustle quickly and have technologies for fast encryption processing (if anyone uses encryption). The disks should not be from the "for home use" series.
3. A lot of RAM. If you are in virtual machines, then 32 GB and higher. Very high quality!!!
4. Certified and high-quality power supply.
5. High-quality cooling of all this.
6. How much electricity will such a tower produce... do you have extra money?
Otherwise: you are simply using ZFS like a hamster, at your own risk.
If you reread on this forum how many problems arose during a "power surge" in the network, how many problems arose during disk degradation and the inability to "simply, quickly and easily" extract IMPORTANT data from a crumbling array, then I would think, is it worth it...
 
I'm having a desktop FreeBSD 14.2 host with 16GB of ram, and recently using zfs.
My windoze-10 vm guest is needing 8GB of ram. This all worked reliably using ufs.
Now the vm keeps pausing. It seems this failure is caused by zfs using up all the memory.
Is changing back to ufs filesystem the best solution ? (All the memory slots are full.)
This humble desktop daily-driver probably doesn't need all the nice fancy features of zfs.
Thanks for any tips or clues.

Do you have a top(1) output of the state where you run into problems?
 
Finally, I recommend a good uninterruptible power supply. If the network is unstable, then a stabilizer and filter before the uninterruptible power supply. And everything will be fine.
 
1. You need more than one or two disks for redundancy and creating an array for unforeseen situations.
2. The disks should be able to rustle quickly and have technologies for fast encryption processing (if anyone uses encryption). The disks should not be from the "for home use" series.
3. A lot of RAM. If you are in virtual machines, then 32 GB and higher. Very high quality!!!
4. Certified and high-quality power supply.
5. High-quality cooling of all this.
6. How much electricity will such a tower produce... do you have extra money?
Otherwise: you are simply using ZFS like a hamster, at your own risk.
If you reread on this forum how many problems arose during a "power surge" in the network, how many problems arose during disk degradation and the inability to "simply, quickly and easily" extract IMPORTANT data from a crumbling array, then I would think, is it worth it...
This can be disputed. In a small installation ZFS can easily run on a single drive (laptop for example) and seems to be more resilient to power loss, than UFS.

Also the term "a lot of RAM" is probably coming from old times when 4GB-s was already a lot of RAM. Today most laptops have "a lot of RAM".

Yes, mirroring in a small system is good, but not required. Personally I have a ZFS mirror of two NVME drives in one of my desktop systems, and this is working and performing very well.
 
1. You need more than one or two disks for redundancy and creating an array for unforeseen situations.
If you want your disk subsystem to be fault-tolerant to one whole disk failing, you need at least two disks. If you want it to be tolerant to double faults (either one whole disk + one unreadable sector on the second disk, or two whole disks failing near simultaneously), you need at least 3 disks. And so on. This is simply an application of information theory: counting bits.

However, even without redundancy, ZFS gives you a lot of other stuff. The single biggest one: checksums. With that, you have a very good chance of catching data corruptions that the disk's ECC doesn't catch, for example user error. You can't fix it once caught, but at least you know about it. Also scrubbing, which makes your disk effectively more reliable, by catching problems early.

In theory, you could even use the "copies=2" feature of ZFS to create redundancy on a single disk. Most people won't do that, because it cuts write speed in half.

2. The disks should be able to rustle quickly
ZFS does not create significantly more disk IO than UFS, so the effect of disk speed is not particularly more significant for ZFS. It is true that the IO pattern of ZFS and UFS are different (in a nutshell, ZFS writes tend to be more sequential, which is both good and bad), so for some workloads ZFS may be faster, and for others it might be slower. The difference between them is not huge.

... and have technologies for fast encryption processing (if anyone uses encryption).
SED or self encrypting disks can encrypt at wire rates. So this is not a factor in performance. Whether disk encryption has value is a difficult question; how hardware encryption at the disk compares to software encryption in the OS is another difficult question. It is quite independent of ZFS.

The disks should not be from the "for home use" series.
In general, if you value your data, you should not use consumer grade disks. But ZFS versus UFS makes little or no difference here. On the contrary, ZFS handles unreliable disks better than most other systems, see above: built-in checksums, scrub, redundancy ...

3. A lot of RAM. If you are in virtual machines, then 32 GB and higher. Very high quality!!!
That canard keeps getting brought up over and over. And it is still false. My home server runs ZFS with 4 disks, and for many years had only 3GB; today it has 4GB. It sadly not have ECC memory (I would like that, but it is hard to get in the form factor I'm after). It runs perfectly fine. The idea that you "need" 32GB of very high quality memory to run ZFS is and remains nonsense.

4. Certified and high-quality power supply.
5. High-quality cooling of all this.
6. How much electricity will such a tower produce... do you have extra money?
ZFS is not particularly more vulnerable to crashes due to power outages. In the old days one could have argued that its log-based design is actually better for crashes; today with both journaling and soft updates in UFS, both are excellent, and very unlikely to lose data on a power outage. UFS may take a little longer to reboot due to fsck, but in practice that's probably not a big deal. While I very much want every user of a stationary computer to have a good UPS, the reason is not ZFS; it is keeping your sanity to not have to have anxiety every time the lights flicker.

And if you want to build a high-performance, high-reliability storage server, you will need multiple disks, a good power supply, and good cooling. Whether you run UFS, ZFS, ext4+RAID, or any other system. If you want industrial-grade storage quality, you will spend industrial-level money and power.

Otherwise: you are simply using ZFS like a hamster, at your own risk.
But if you run another file system on the same hardware, your risk will not be better; it might be worse.

I see the tradeoff differently. Yes, ZFS uses more CPU power to do IO. Its cache management algorithm may be more reluctant to release RAM, which may lower effective machine performance if other RAM hungry and inflexible tasks (such as a GUI or VMs) are running on the same machine. It is harder to learn and administer for Unix old-timers, since you have to learn new concepts (such as pools and parameters) and commands (not just mkfs and mount). The benefit you get for the slightly higher CPU usage and perhaps more memory contention is significantly better reliability, even with a single disk. I think on a server machine, ZFS will de facto always win reasonable tradeoffs, assuming the admin can be taught how to use it. On a pure GUI machine (laptop), if the on-disk data is considered transient anyway, and the user is willing to reinstall the OS regularly (perhaps because they are distro hopping in the first place), ZFS may not be worth the effort.

My personal opinion: If it weren't for ZFS, I would not be running FreeBSD. And no, I've never been a ZFS developer, so I don't have any "parentage" feeling for it; matter-of-fact, I've worked on several storage software stacks that compete with ZFS.
 
ZFS and RAM: primary use of memory is for ARC (Read cache). If the workload is primarily reads of a lot of different files I think that is what triggers most of the RAM usage.
My daily usage, which is "generalized desktop usage" I've not seen difference in performance between UFS and ZFS. My primary reason for using ZFS, even on single disk systems is Boot Environments. BEs take a lot of the angst when doing system upgrades. Manually create one before doing quarterly packages so if something breaks, roll back. Let freebsd-update create one, if something breaks, roll back.
The power supply in any system is critical; if you are building your own you can choose, buying a consumer grade PC from anywhere and the power supply is barely adequate.
 
I'm having a desktop FreeBSD 14.2 host with 16GB of ram, and recently using zfs.
My windoze-10 vm guest is needing 8GB of ram. This all worked reliably using ufs.
Now the vm keeps pausing. It seems this failure is caused by zfs using up all the memory.
Is changing back to ufs filesystem the best solution ? (All the memory slots are full.)
This humble desktop daily-driver probably doesn't need all the nice fancy features of zfs.
Thanks for any tips or clues.

Also really make _sure_ it's not the disk that silently is starting to have problems. Check with 'smartctl' and do some tests...

(It's pretty common for SATA drives to silently start to show sudden "pauses"/delays when it starts to develop problems and has to retry reading stuff and reallocate sectors and stuff...)

But I'd try limit the zfs arc size to something like 4GB if you are using 8GB vms on a 16GB RAM machine.
 
Back
Top