UFS Detecting Periodic Trim of SSD under UFS

I

Iggy64

I have begun migrating my three laptops from Devuan to FreeBSD. After reviewing the very helpful user manual and other readily available resources, I got one laptop (Dell Latitude E6400) up and running a couple of months ago. It is a single-user machine running a single 500GB SSD. I have it set up with a single UFS partition (using a file for swap) running soft updates but not continuous trim.

I use the machine for building wiki-type knowledge bases for various clients. My key apps are therefore Cherrytree and Firefox. My user data is saved out to USB drives, which are formatted exFAT to be shareable with Linux and Windows machines.

Over my many years running Linux, I have typically used fstrim to do periodic trimming of my system SSD. Although there is no direct analog of fstrim in FreeBSD, I note that an -E flag was some time ago added to fsck_ffs, to permit on-demand trimming of the system drive. Based on suggestions found in this forum and elsewhere, I have tried running

#fsck_ffs -Ed /dev/ada0s1a

in single-user mode. The -d flag is added in order to generate debugging output, again at the suggestion of forum contributors. Hopefully, this would provide some feedback on the trim operation.

The command runs successfully, and announces I have a clean file system. However, I see nothing in the output that hints at any trim function output. (The actual output is too voluminous to post here.) I am looking for some mention of reclaiming unallocated blocks, or something similar. I have not found any published examples of what to expect in the debugging output.

Can anyone educate me on how I might learn what, if any, trimming has been done using the fsck_ffs command?

I have been researching this and experimenting for the past week, but am running out of ideas.

The documentation behind FreeBSD is so darn thorough and diverse, I have had no need to post questions to the forum until this issue. Sorry I haven't resolved it on my own.
 
Running:
fsck_ffs -E diskdev
should issue a TRIM to the underlying device for all free blocks in the filesystem each time fsck_ffs -E is run, but only if the filesystem had TRIM enabled via the "-t" option of newfs(8) or tunefs(8).

TRIM might be ignored when an SSD is connected via a USB dongle -- the entire chain of kernel => USB driver => dongle => SSD would need to properly support and pass TRIM all the way through.

Most modern SSDs can do TRIM operations exceptionally fast, (as little as a few seconds to TRIM an entire <= 2TB SSD, especially if the TRIM'd blocks were already "pre-scrubbed" on the SSD).

The only way I have been able to verify TRIMs are happening is to run:
systat -iostat -numbers -- 1
in a different window/terminal, then run fsck-E device -- if at any point during Phase 5 of fsck_ffs you see the systat KB/t column for the fsck-device show a number much bigger than 128 (e.g.: 131072), it's a telltale that TRIM was used (note that 131072 is an unrealistic KB per disk I/O that would ever show for normal disk I/O -- KB/t is usually only 0-128); I have only ever seen a KB/t that high during TRIM). This "telltale" has worked reliably for me for FreeBSD 10-14. YMMV.

If you use systat, I would recommend not including the "-d" flag on fsck_ffs, because the extra fsck debug printf's might slow-down the clearing of blocks enough that systat might not show 131072 KB/t, which clearly differentiates TRIM from regular I/O. And FWIW, I looked at the fsck_ffs sources, it does not look like "-d" would tell you anything specific about TRIM anyway.

if you have TRIM enabled on the filesystem, the kernel will issue a TRIM for every freed block as each free happens, so I doubt there is any need to run fsck_ffs -E periodically, unless you just enabled TRIM and want to make sure that all free-but-previously-unTRIM'd-blocks are now TRIM'd.
 
Thanks gpw928 for the additional reference.

In reply to M F, thank you for your perspective on SSD trim. It may be that I am misinterpreting a man page or two:

It was my impression that using tunefs with the "-t" flag would initiate dynamic (on-the-fly) trim. This is not my preferred method, as it creates lots of small scattered chunks.

So I looked for a method to do periodic trim and found the fsck_ffs -E approach, which lets me collect reallocated blocks over time. Its man page says

"This is useful for filesystems which have been mounted on systems without TRIM support, or with TRIM support disabled"

This suggests that dynamic trim via tunefs should be turned off to prevent immediate reallocation of blocks, in order to accumulate trim into bigger batches. That is, if I use tunefs -t, blocks will be immediately freed and nothing will accumulate for periodic trimming.

But perhaps I am misinterpreting the whole situation. I am grateful for your explanation, and I will dive back into this issue with a more-open mind about tunefs -t. I will run some additional tests to see if I have misinterpreted the man pages, and I will check back in with my findings.
 
Ahhhh, I peeked at the fsck_ffs source code again -- I believe you are correct that fsck_ffs -E will issue TRIM for all free filesystem blocks even if the filesystem's TRIM is not enabled -- thank you for pointing-out that subtlety.

I still think it's better to enable filesystem TRIM via tunefs -t, and let the OS tell the SSD to TRIM each free'd block as each free happens, because the SSD's TRIM-block-scrub has to be done sooner or later anyway. If you defer it by disabling TRIM, then if/when the SSD's pool of pre-scrubbed blocks ever gets depleted (likely when the SSD is nearly-full and there is a flurry of writes), the write performance of the SSD goes into the toilet (I measured roughly 1/10 of normal) because in that case the SSD has to scrub blocks live/on-the-fly in order to fulfill each write. Feel free to ask me how I know this if you really want the gory details. :)

Also, I believe SSDs "scrub" blocks in fixed-size chunks, serially, one block at a time. I think SSD manufacturers refer to this as the "erase block size" (EBS). On the drives I use (Crucial), the EBS is 4K bytes.

If filesystem TRIM is enabled, the SSD can scrub TRIM'd blocks while it's "idle", resulting in maximal instant-availability of blocks for reallocation.

Bottom line, I can't see any advantage to allowing a ton of un-TRIM'd blocks to accumulate and then TRIM'ing them "in bulk" with fsck -E based on my understanding of how TRIM works. And the potential performance-hit of not doing TRIM ASAP can be very significant, although other manufacturer SSDs might behave/perform differently. YMMV.
 
Thanks very much for all this additional info and context. It will no doubt help me resolve my quandry.

(In reality, this is a somewhat minor issue in my present use case, as my SSD is WAY overprovisioned; I do not store/delete many user files on the 500GB SSD -- they are stored externally on rotating disks. However, gaining knowledge on this topic may well help me in other use cases.)
During my many years in the Linux world, I have been encouraged to use periodic trim rather than continuous (dynamic) trim. The fstrim script was designed specifically for that purpose, in preference to the " discard" flag in /etc/fstab. Some example references:

Code: 
https://forums.FreeBSD.org/threads/detecting-periodic-trim-of-ssd-under-ufs.95422/
Ubuntu enables periodic TRIM by default [2], Debian does not recommend using continuous TRIM and Red Hat recommends using periodic TRIM over using continuous TRIM if feasible

Code: 
https://www.unitedbsd.com/d/670-trim-support/4
nortonham wrote: The first consideration to make is that while TRIM will most likely improve write performance by allowing preemptive garbage collection (so files can be directly written without having to reclaim free pages), it is not completely devoid of downsides. In fact, when acting in background (as a static mount option), TRIM may also lead to a paradoxical increased write latency, or even lead to data corruption on certain known products, whose firmware either misreports support for queued TRIM or has critical bugs in its implementation. For these reasons, TRIM may be more safely and efficiently implemented ad hoc by issuing the command to discard unused blocks during periodical maintenance, rather than leaving it persistently active as a mount option, which forces it to be invoked after every single file deletion. On Linux this is achieved with fstrim(8) and can be automatized via cron jobs.

In general, then, I have been encouraged to gravitate to a weekly trim of my SSD. However, it is clear that "one size does not fit all." It depends on your use case. For example:

High-write-activity systems may benefit from Continuous Trim (to reduce fragmentation), while low-write-activity systems may be better suited for Periodic Trim (reduced load on system resources).

Regardless of which way I end up going, I am still quite curious about any means for determining whether my trim strategy is actually trimming anything. For example, in the Linux world, the old fstrim script provides a nice log of amount of trimming performed.

Thanks, again, for all your great input. It is helping me think this through much more thoroughly.
 
Some cheap consumer grade SSDs may not have sufficient over-provisioning to perform well under write-duress when full (as the firmware struggles to have sufficient pre-zero'd pages ready to deploy for writing).

Enterprise class SSDs have a lot more over-provisioning than consumer grade SSDs. This improves their performance under write-duress, and also improves their longevity (as "bad blocks" develop, and have to be retired).

You can manually increase the over-provisioning pool by TRIMing a portion of your SSD and never deploying it for use by the O/S. This is likely to help performance in massively write-intensive situations.

There's a lot of support out there for periodic trim, as it reduces both overhead and fragmentation. For example the ArchWiki says of TRIM:

Note: If you want to use TRIM, use either periodic TRIM or continuous TRIM. Continuous TRIM is not the most preferred way to issue TRIM commands among the Linux community. For example, Ubuntu enables periodic TRIM by default [2], Debian does not recommend using continuous TRIM and Red Hat recommends using periodic TRIM over using continuous TRIM if feasible [3].​
 
Thanks gpw928 for the additional commentary. It builds further useful perspective around the issues being discussed here.

Thanks, too, to all who kindly contributed their experience and ideas regarding this issue of SSD trim under UFS. Combined vast and well-organized documentation behind FreeBSD, this forum seems a friendly and constructive source of support when details need to be ironed out.

My original question, though, is not yet completely resolved. That is -- how can I determine whether the trim commands I give are actually working as expected?

Under Linux, fstrim can be run as a single command, or run as a cron job to do periodic trim. This script sends an output log to /var/log/trim.log. It shows the date, time, and amount of trimming done. I'm not seeking anything that elaborate here, but merely some way of getting feedback from the system that I am, in fact, actually accomplishing the trim I am trying to do.

M F gave me one suggestion, based on his experience, for running

systat -iostat -numbers -- 1

before and after my trim command. I am going to experiment with this a bit and see if I get good results I can understand at my level of expertise.

Meanwhile, any additional ideas on how to detect/quantify the results of trim commands would really help my education in this area.

Again, thanks to all for the very helpful discussions.
 
I just tested using FreeBSD 14.1, now I recommend using 10,000 as the "treshold" KB/t value to determine TRIM vs not (instead of 128 that I said in my first post this thread, because now it appears that regular I/O on modern FreeBSD can go up to 1024 KB/t).

When I run fsck_ffs -E on a 500GB Crucial SSD where the filesystem is mostly free space, I see about 8 seconds where systat displays KB/t numbers in excess of 100,000. Less free space on your filesystem would mean fewer #secs with high KB/t because fewer blocks to clear means less time. If you have almost no free space in your filesystem, it's possible the KB/t number might not spike high-enough even during 1 second to be "definitive".

To maximize the number of rows displayed by systat, once you are running:
systat -iostat -numbers -- 1
I recommend typing:
:only ada0
and press <ENTER> (note the leading-colon). This tells systat to display stats only for /dev/ada0, to maximimize the number of rows will stay on the screen with 1-second intervals.

The only reason I can think-of why fsck_ffs -E might not issue TRIMs is if your specific SSD has what is called a "quirk" entry in the disk driver to disable TRIM (I think a quirk will show-up in /var/run/dmesg.boot on the probe-line for your device with the string TRIM_BROKEN). This quirk is added by FreeBSD developers for specific SSD make/models when there are bug reports that TRIM is known to be broken, and therefore FreeBSD disables it, figuring that it's better to disable than risk corrupting data.
 
Thanks very much, M F. Immediate circumstances force me to wait a couple of days before I will have a chance to explore this without distractions.

FYI, the SSD on the machine in question is a 500GB Crucial MX500. Since it contains only the base system and several user apps, usage is only 15%. On Debian/Devian-based systems, fstrim runs well and reports reasonable results to /var/log/trim.log.

Once I get the chance to test the systat approach, I'll report back here on what I learn.

I really appreciate the support.
 
You must log in or register to reply here.
Back
Top