Solved rsync blocks zfs operations and blocks input devices, unusable

L

loveydovey

So I start a simple rsync from a USB drive to my zroot user home folder. It works great.
But then, while rsync is copying, if I try to import a ZFS pool from another device, it just gets stuck at "importing ZFS pool" and never completes it (although it seems to import it in the background after I Ctrl+C), and if I try to export ZFS pool, same story, gets stuck at exporting ZFS pool.
If I try to work with an imported ZFS pool while rsync copy is ongoing, it doesn't let me do I/O very much (browser stops working).
Worse, it almost hangs for a while every so often - the mouse and keyboard stop working completely, then start working again.
The system just becomes unusable.
I tried "nice -n 20" the rsync copy operation, same problem - didn't solve it.
Don't know what to do. I'm scared to copy file snow 😭😭😭
 
Sounds like FreeBSD doesn't have the I/O system like Linux. No `ionice`. The only thing that helped was using --bwlimit to cap the I/O bandwidth during copy (after I also lowered zfs dirty data max to 64MB, which did not work alone).
It feels very crude. Is that the way to handle this?
 
Can confirm that this affects you only if I copy into my zroot pool's dataset. So topping out writes to inside of the zroot pool is what causes this issue.
When I write to an external drive without speed limits, things work fine.
Is that all that's needed to nearly hang a FreeBSD server? Max out writes to zroot? *gulp* 😰
 
in all likelihood the problem is in the OP's expectations — writing to a mostly full zpool on decade-old hardware is going to suck, but the problem there does not lie within freebsd
 
I'm migrating from Linux. So it was an ext4 partition on an external USB3.0 drive that was getting copied into zroot/home/user on an SSD. Nothing else was being done pretty much..and it really wouldn't let me do anything much.
What I just realized too is that I had zfs copies=2 set on an entire zroot/home/user (as well as on the entire pool except pool's root because when I set copies=1 zroot, it was not done recursively), so this may overdosed zroot on writes. Maybe someone could try to replicate this?
 
you are describing a disk io problem and have mentioned the disks 0 times, shown 0 information about your zpool, have not named the kind of CPU or amount of RAM in the machine, all you have to say is that it used to run linux. and yet we're the unhelpful one? ok
 
You are brandishing a soliloquy about pronouns in your signature, that's not helpful (per US government). And asking about irrelevant information because CPU and RAM are a non-issue, or I would've volunteered that right away.
 
so you have nothing to say about the disks you are using, or how full your pool is, or any actually useful information about troubleshooting, gotcha
 
loveydovey said:
I had zfs copies=2 set on an entire zroot/home/user
Click to expand...
Copies=2 was not an issue. I just tried it with copies=1, and it still freezes the system when I write to zroot/home/user at max write speed.

I just don't understand why a user can freeze his freshly imbued with FreeBSD machine purely by starting to copy files to his home folder on zroot.

Must I manage the kernel's I/O myself instead of expecting the kernel to manage its own I/O? Is that sort of the FreeBSD idea? Must I become the kernel myself? 😱

I'm trying to understand this from the design point of view.
 
`atop` shows bursts of writes to my zroot SSD ranging from 50-85 MB/s or so. This is without limiting rsync bandwidth.
The freeze resolves when I cap rsync's bandwidth to 30 MB/s.
 
AI says this is a side effect of FreeBSD and ZFS defaults being tuned for Server Throughput, whereas Linux (desktop distros) defaults are tuned for Interactive Latency.
True or False.
 
if you specify a repeatable test, others can try it out and may be suggest alternatives. Also, provide some specific details about you machine: cpu, speed, number of cores, memory size, df, etc. even if you think they’re not relevant. Also: freebsd is not linux. You have to learn its peculiaries to use it effectively.
 
bakul said:
if you specify a repeatable test, others can try it out and may be suggest alternatives. Also, provide some specific details about you machine: cpu, speed, number of cores, memory size, df, etc. even if you think they’re not relevant. Also: freebsd is not linux. You have to learn its peculiaries to use it effectively.
Click to expand...
i7-4700mq (8 core), 32GB DDR3 RAM. FreeBSD 14.3. I repeat myself, but it's an rsync copy from an external USB3.0 enclosure of an HDD drive over USB3 to an internal SATA SDD drive with zroot on it, copying to zrooot/home/user. Zroot on SSD is nearly empty when I start rsync. Zroot has mostly defaults, has atime off, xattr sa, compression is lz4.

I know I'm an early adopter of FreeBSD, but maybe, just maybe, there is a future for this.
 
You must log in or register to reply here.
Back
Top