Other F2FS in FreeBSD

YuryG · Aug 10, 2023

Is there a way to mount at least in read only mode a flash drive with F2FS file system? May be using Fuse or even Linux emulation mode? Could not find in ports anything appropriate.

ralphbsz · Aug 11, 2023

To my knowledge, there is no F2FS implementation for any OS other than Linux (and Android). You could try running Linux (perhaps in a VM) and serving it from there, via a protocol like SMB or NFS.

cracauer@ · Aug 11, 2023

There should be a rule that everybody who makes a new filesystem native to some OS also supplies a medium-performance fuse module to access it.

ralphbsz · Aug 11, 2023

For single-node single-disk file systems, that would be a reasonable engineering technique. In particular since such a userspace module can be helpful in debugging things, for example metadata problems. Perhaps not a complete fuse implementation, but at least a userspace tool that can access metadata and data.

For cluster- and planet-scale file systems, and multi-disk file systems, it's way too complicated to encapsulate them in something like fuse.

kpedersen · Aug 11, 2023

exFat is meant to be fairly "optimized" for flash drives. Could this be an option?

The FUSE implementation is actually pretty good. Faster than NTFS.

smithi · Aug 11, 2023

kpedersen said:
exFat is meant to be fairly "optimized" for flash drives. Could this be an option?

Ventoy uses exFAT by default for its main storage for installable .ISO and .img files. Works fine, without FAT32 filesize limitations.

kpedersen said:
The FUSE implementation is actually pretty good. Faster than NTFS.

Yes. FreeBSD port | package:

sysutils/fusefs-exfat
sysutils/exfat-utils

balanga · Aug 12, 2023

smithi said:
Ventoy uses exFAT by default for its main storage for installable .ISO and .img files. Works fine, without FAT32 filesize limitations.

I had numerous problems, although don't recall the specifics, when I first installed Ventoy using exFAT. When I changed to NTFS they went away. I may try another install if you find it reliable.

In fact, I was just looking through a micro-SD card unaware of which filesystem was being used. Couldn't mount it as ntfs or msdosfs, so installed fusefs-exfat which appeared to provide access, but a number of files in DCIM/Camera were not shown under FreeBSD, although Windows had no problem.

YuryG · Aug 12, 2023

kpedersen said:
exFat is meant to be fairly "optimized" for flash drives. Could this be an option?

The FUSE implementation is actually pretty good. Faster than NTFS.

Yes, I have one Flash drive in exFAT, supported better than F2FS (the latter is supported only on OpenWRT Linux-based router, while the former I can use on almost all my devices). Although I was amused that none of my home Android devices support F2FS (OTG), some newer smartphones support exFAT. And our older home Android-TV supports only FAT32 and NTFS(!) flash sticks.
What a mess! )

smithi · Aug 12, 2023

balanga said:
I was just looking through a micro-SD card unaware of which filesystem was being used. Couldn't mount it as ntfs or msdosfs, so installed fusefs-exfat which appeared to provide access, but a number of files in DCIM/Camera were not shown under FreeBSD, although Windows had no problem.

Do you mean some files in DCIM/Camera - presumably from an Android phone - didn't show up in a directory listing, or that they didn't display as photos in the program you were using?

balanga · Aug 12, 2023

No files were shown under DCIM/Camera on FreeBSD

Bash:

root@M73:/media# gpart show da1
=>       63  124735425  da1  MBR  (59G)
         63      32705       - free -  (16M)
      32768  124702720    1  ntfs  (59G)

root@M73:/media# file -s /dev/da1                                           
/dev/da1: DOS/MBR boot sector; partition 1 : ID=0x7, start-CHS (0x2,10,9), end-CHS (0x3ff,254,63), startsector 32768, 124702720 sectors, extended partition table (last)
root@M73:/media# file -s /dev/da1s1
/dev/da1s1: DOS/MBR boot sector
root@M73:/media# mount.exfat /dev/da1s1 /mnt
FUSE exfat 1.4.0 (libfuse2)
WARN: volume was not unmounted cleanly.
root@M73:/media# ls -al /mnt
total 2185
drwxrwxrwx   1 root  wheel  131072 Jan  1  1970 .
drwxr-xr-x  20 root  wheel      26 Aug 10 13:01 ..
drwxrwxrwx   1 root  wheel  131072 Mar 10  2022 .android_secure
-rwxrwxrwx   1 root  wheel       0 Feb  4  2023 .fe_tmp
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Alarms
drwxrwxrwx   1 root  wheel  131072 May 10  2022 Android
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Audiobooks
drwxrwxrwx   1 root  wheel  131072 Mar 19  2022 DCIM
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Documents
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Download
drwxrwxrwx   1 root  wheel  131072 Mar 10  2022 LOST.DIR
drwxrwxrwx   1 root  wheel  131072 Mar 10  2022 Movies
drwxrwxrwx   1 root  wheel  131072 Mar 10  2022 Music
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Notifications
drwxrwxrwx   1 root  wheel  131072 Mar 10  2022 Pictures
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Podcasts
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Recordings
drwxrwxrwx   1 root  wheel  131072 Mar 31  2022 Ringtones
drwxrwxrwx   1 root  wheel  131072 Feb  4  2023 System Volume Information
root@M73:/media# ls -al /mnt/DCIM
total 384
drwxrwxrwx  1 root  wheel  131072 Mar 19  2022 .
drwxrwxrwx  1 root  wheel  131072 Jan  1  1970 ..
drwxrwxrwx  1 root  wheel  131072 Feb  4  2023 Camera
root@M73:/media# ls -al /mnt/DCIM/Camera
total 0
root@M73:/media#

Windows shows around 20 jpgs in this directory.

chungy · Aug 12, 2023

Try running chkdsk /f X: on your Windows computer (with X: as the drive letter the SD card shows up as)

You can try using exfatfsck from sysutils/exfat-utils, but since you have access to Windows, it's probably best to use Microsoft's tool.

balanga · Aug 12, 2023

chungy said:
Try running chkdsk /f X: on your Windows computer (with X: as the drive letter the SD card shows up as)

You can try using exfatfsck from sysutils/exfat-utils, but since you have access to Windows, it's probably best to use Microsoft's tool.

I tried running exfatfsck /dev/da1s1

ERRORS FOUND:2, FIXED: 0.

Still no sign of the jpgs which Windows sees.

YuryG · Aug 17, 2023

kpedersen said:
exFat is meant to be fairly "optimized" for flash drives. Could this be an option?

The FUSE implementation is actually pretty good. Faster than NTFS.

Well, recently I've encountered a problem (bug?) using exFAT formatted flash stick. While transcoding a 9GiB file from it via ffmpeg I've got "read error" around 8GiB boundary (repeatedly, on the same timecode near the end of file). Checked and found no errors on the device. Also on Windows system ffmpeg had no "read error" with the same drive in that file position. ;(

smithi · Aug 17, 2023

YuryG said:
Well, recently I've encountered a problem (bug?) using exFAT formatted flash stick. While transcoding a 9GiB file from it via ffmpeg I've got "read error" around 8GiB boundary (repeatedly, on the same timecode near the end of file). Checked and found no errors on the device. Also on Windows system ffmpeg had no "read error" with the same drive in that file position. ;(

Have you reported this to the port maintainer [edit: samm@FreeBSD.org] yet?

Given that it's reproducible, you should have enough data for a PR on it?

smithi · Aug 17, 2023

balanga said:
root@M73:/media# mount.exfat /dev/da1s1 /mnt FUSE exfat 1.4.0 (libfuse2) WARN: volume was not unmounted cleanly.

balanga said:
root@M73:/media# ls -al /mnt/DCIM/Camera total 0 root@M73:/media#

It seems that you ignored the warning about not having been unmounted cleanly, before then finding files missing?

I would have first umount'd the disk then run exfatfsck on the unmounted disk - fsck is not (generally) supposed to write to (that is, fix) a mounted disk.

balanga said:
I tried running exfatfsck /dev/da1s1

ERRORS FOUND:2, FIXED: 0.

Still no sign of the jpgs which Windows sees.

Again, it appears that the disk was still mounted (errors found, not fixed), so you'd expect no change.

Try running exfatfsck again on the unmounted disk; it may find and fix those 2 errors.

balanga · Aug 17, 2023

smithi said:
It seems that you ignored the warning about not having been unmounted cleanly, before then finding files missing?

I would have first umount'd the disk then run exfatfsck on the unmounted disk - fsck is not (generally) supposed to write to (that is, fix) a mounted disk.

Again, it appears that the disk was still mounted (errors found, not fixed), so you'd expect no change.

Try running exfatfsck again on the unmounted disk; it may find and fix those 2 errors.

Tried again as you suggested...

Code:

root@X1:~ # umount /mnt/exfat
root@X1:~ # mount.exfat /dev/da0s1 /mnt/exfat
FUSE exfat 1.3.0
root@X1:~ # ls -al /mnt/exfat/DCIM/Camera/
total 0
root@X1:~ # umount /mnt/exfat
root@X1:~ # exfatfsck /dev/da0s1
exfatfsck 1.4.0
Checking file system on /dev/da0s1.
File system version           1.0
Sector size                 512 bytes
Cluster size                128 KB
Volume size                  59 GB
Used space                  386 MB
Available space              59 GB
ERROR: 'IMG_20220819_153055.jpg' is empty but start cluster is 0x9a8.
ERROR: 'IMG_20220819_153055.jpg' is empty but marked as contiguous (0x20).
Totally 582 directories and 207 files.
File system checking finished. ERRORS FOUND: 2, FIXED: 0.
root@X1:~ #

YuryG · Aug 17, 2023

smithi said:
Have you reported this to the port maintainer [edit: samm@FreeBSD.org] yet?

Given that it's reproducible, you should have enough data for a PR on it?

Not yet. Considering the option. Shall have to find really reproducible case I could post.

chungy · Aug 18, 2023

Try checking it in Windows. Microsoft's chkdsk probably does a lot more than exfatfsck.

YuryG · Aug 18, 2023

chungy said:
Try checking it in Windows. Microsoft's chkdsk probably does a lot more than exfatfsck.

Done this and test ended without finding any errors (although I am not sure if this really means Windows did not modified the filesystem on a stick during “test”). But the error persists in FreeBSD when watching or transcoding the file mounted using fuse, and not in Windows.

smithi · Aug 18, 2023

YuryG said:
Not yet. Considering the option. Shall have to find really reproducible case I could post.

Could you copy that 9G file to another same-size stick to confirm it's not just a bad stick (they happen). Also to see if diff(1) finds anything.

That 8G point does sound suspicious, can you be more precise about size, in bytes?

Sure sounds like enough to email the maintainer [urge!]

YuryG · Aug 18, 2023

smithi said:
Could you copy that 9G file to another same-size stick to confirm it's not just a bad stick (they happen). Also to see if diff(1) finds anything.

That 8G point does sound suspicious, can you be more precise about size, in bytes?

Sure sounds like enough to email the maintainer [urge!]

Well, again checked and copied with a Windows 7 notebook (on a slow USB2 port) — without errors.

And trying to copy the same file on my FreeBSD desktop with USB3 port gives “Input/output error” leaving 7722762240 (=0x1,CC50,0000) bytes copied out of 9454933972 bytes.

dmesg says nothing after flash stick insertion
“umass0: SCSI over Bulk-Only; quirks = 0x8100
…
da0: quirks=0x12<NO_6_BYTE, NO_RC16>”.

smithi · Aug 18, 2023

YuryG said:
Well, again checked and copied with a Windows 7 notebook (on a slow USB2 port) — without errors.

Copied to a different stick?

YuryG said:
And trying to copy the same file on my FreeBSD desktop with USB3 port gives “Input/output error” leaving 7722762240 (=0x1,CC50,0000) bytes copied out of 9454933972 bytes.

Sorry, for absolute clarity to rule out a bad stick: copy the same file to a different or same stick as before?

Thanks for hex, noticing the 64kiB boundary - I assume using dd(1) with bs=64k ?

So it's nowhere near 8GiB as a programming error factor.

YuryG said:
dmesg says nothing after flash stick insertion
“umass0: SCSI over Bulk-Only; quirks = 0x8100
…
da0: quirks=0x12<NO_6_BYTE, NO_RC16>”.

Unfamiliar with NO_RC16 quirk, OWS common enough.

If it's a different stick I think you have enough data ...

YuryG · Aug 18, 2023

smithi said:
Copied to a different stick?

Sorry, for absolute clarity to rule out a bad stick: copy the same file to a different or same stick as before?

Thanks for hex, noticing the 64kiB boundary - I assume using dd(1) with bs=64k ?

So it's nowhere near 8GiB as a programming error factor.

Unfamiliar with NO_RC16 quirk, OWS common enough.

If it's a different stick I think you have enough data ...

Copied from this stick to a hdd.

Copied with "cp -v /mnt/flash/file /usr/home/"

Haven't used another USB flash stick for exFAT testing yet. For now just the same stick working in Windows and erroring on FreeBSD.

smithi · Aug 19, 2023

YuryG said:
Copied from this stick to a hdd.

Copied with "cp -v /mnt/flash/file /usr/home/"

Ah, so the 64kiB size at the error might be significant.

YuryG said:
Haven't used another USB flash stick for exFAT testing yet. For now just the same stick working in Windows and erroring on FreeBSD.

I guess only trying it with another stick will confirm whether that stick or the exfat fuse module has the problem?

Either way, it couldn't hurt to point the maintainer to this thread? I will if you're shy ..

YuryG · Aug 19, 2023

smithi said:
Ah, so the 64kiB size at the error might be significant.

I guess only trying it with another stick will confirm whether that stick or the exfat fuse module has the problem?

Either way, it couldn't hurt to point the maintainer to this thread? I will if you're shy ..

I've put a bug report

PR 273201