Solved rsync blocks zfs operations and blocks input devices, unusable

[ralphbsz]

I know I'm an early adopter of FreeBSD

That's funny right there.

BSD has existed for about 40 or 50 years. FreeBSD for about 25 or 30. Quite a few people on this forum have run it for that long. I'm only at about year 15 (although I used other BSD-derived OSes in the 1980s).

Your post here is still completely unhelpful.

Here would be my suggestion if you want help debugging this: First, turn of all GUI; perform the same operations from the CLI. Make sure no other processes are running. While doing these operations, do some monitoring, like some combination of top, vmstat and iostat. Save the output of those. And then post exactly what you were doing, and how many IOs were actually running, and what the memory pressure is. You might be having an IO scheduling problem, or you might have any other problem.

 

[cracauer@]

I still can't figure out what filesystems are involved. Is there a second ZFS? What about the ext4 mentioned on the first place. Is the rsync target an internal SSD or also USB?

 

[loveydovey]

You are all not helpful.

Specific solution: Disable the sync on the dataset

rsync tries to be safe. It often asks the filesystem: "Are you SURE this is written to disk?" (fsync).
When ZFS receives this request, it must write to the ZIL (ZFS Intent Log) immediately.

You can tell ZFS: "Ignore the safety checks. Just cache it in RAM and write it when you can."

The Fix:
Set this on the Target Dataset (where you are copying TO):

zfs set sync=disabled zroot/your/target/dataset

(Warning: If the power cuts out during the copy, you lose the last 5 seconds of data. For a home copy, this is usually acceptable).

All this tells us that IOPS (Input/Output Operations Per Second) and Latency, not bandwidth, are the culprit.

Keeping sync disabled is usually not a problem. You might lose some files within the 5 seconds before abrupt power loss and your browser profile might get corrupted. But definitely not desirable to disable sync on the entire zroot, although will not corrupt your system.

Or does it make sense to disable sync on the entire zroot home folder permanently for desktop users? It's such a small fix with little risks that prevents freezing/locking up to your entire machine during write intensive tasks that are so prevalent for desktop users who would most likely write a lot into their home folders.

[By the way, nothing else worked, I tried so many different things from rtprio to kernel zfs settings, etc etc; only disabling sync works, and it makes sense now too]

 

[bakul]

if you “dd if=usb-device bs=1m of=/dev/null” while doing lots of it on your ssd zfs, do you see a similar slowdown or is it fast?

You can also play with hw.usb.xhci.use_polling and couple other sysctl to see if things improve. You can also play with scheduling parameters such as sysctl kern.sched.steal_thresh…. Websearch for settings for better interactive performance on freebsd. Or ask Gemini!

 

[bakul]

You are all not helpful.

People who argue while asking for help don’t help their cause. Better to acknowledge others who take time to respond & are trying to help you even if you don’t find their suggestions helpful.

bursts of writes to my zroot SSD ranging from 50-85 MB/s or so

You can use zpool iostat 3 for ex to get iostats reported every 3 seconds. What is the raw speed on your ssd? 50-80 MBps seems a bit low. I guess this is limited by your usb hdd?

By the way, nothing else worked, I tried so many different things from rtprio to kernel zfs settings, etc etc; only disabling sync works

Your immediate problem may be solved but I suspect this works around the underlying cause….