"Best" (i.e. "universal") filesystem for portable USB drives?

I have a couple of 8Tb drives that I connect to my PC, via USB adapters, just to back up my daily data. I also routinely use USB "thumb" drives as well.

I'm looking for a recommendation for a filesystem that is "universal" in the sense that I can take the USB drives and routinely move and connect them to FreeBSD, Debian and Microsoft Windows PCs. I "think" the best filesystem would be exFAT, since all three OSes could easily read/write to a drive with that filesystem.

Suggestions? Opinions?

Thank you,
Ed
 
I "think" the best filesystem would be exFAT, since all three OSes could easily read/write to a drive with that filesystem.
The other alternative is FAT(32), but exFAT is probably the better choice given the limitations of FAT (max 4 GB file size for example). But FAT can be read by a barebones FreeBSD install, for exFAT you need to install additional ports/packages.
 
The other alternative is FAT(32), but exFAT is probably the better choice given the limitations of FAT (max 4 GB file size for example). But FAT can be read by a barebones FreeBSD install, for exFAT you need to install additional ports/packages.
Thank you !

Ed
 
I second what SirDice said. The FAT filesystem are the first choice, when it comes to be used most easily on different platforms including such things like mediaplayers, TV sets etc. While FAT32 is a tiny bit more compatible while exFAT has fewer limitations. But while attaching to desktops machines or servers, only, that's to neglect - prefer exFAT then.

Anyway, since I always like to throw in questions that may question a whole idea completely 😁:cool::
I simply cannot judge the very overall situation you have, and what you use your drive(s) for, including mobility or else. But when I read "8TB drives and USB thumb drives" in one sentence, I simply wonder if it wasn't a better idea to create a NAS instead: ZFS raid pool, other machines connect via LAN due NFS or Samba to it, instead of having a bunch of single storage drives.
No need to answer, nor to explain, especially no "excuses", just for you maybe didn't thought of it yet, and if it could suit your situation (I don't know), perhaps maybe could be just an idea.
 
I cleansed Windows from my porch. On sensitive HDDs and Thumb drives I use LUKS/ext4.

Using ZFS sounds interesting. Even the encryption should work better, at least no cryptsetup needed. And you don't have to encrypt the whole disk, just the dataset/partition alas no GUI integration - automatic mount will fail.
 
I'd stay with FAT32. Can a XP system or a mp3-player with a file manager read exfat and put files on it? I never used it. What does a system from before exfat think it is?
 
FAT32 isn't suitable for backups, but for thumbdrives or sdcards it's perfect imo.
I haven't found any OS or device, that doesn't support it in 20 years or more.
FreeBSD, Linux, Windows, mobile phone and cameras can write to it out of the box.
All mp3 players and car stereos I've owned so far where able to read it, too.
It does not support UTF-8, but that's acceptable for me.
 
Thank you all for your input.

I think I know the answer, but I'll ask anyway...

If I'm using the entire drive for storage only, I don't need to create a GPT partition table at all and I don't need to create any partitions, correct? In other words, I can simply:

sudo mkexfatfs -n volume_name /dev/daX

and call it good, right?

Ed
 
If I'm using the entire drive for storage only, I don't need to create a GPT partition table at all and I don't need to create any partitions, correct? In other words, I can simply:
Ah, no.
AFAIK ZFS is capable to implement whole drives directly into its pools, but if you use some filesystem like UFS, FAT or something like that, you first have to create a partition table and then at least one partition. But with
gpart that's two short lines and two seconds in the shell. The newfs you run afterwards will take some minutes on a 8TB drive - but it's not rocket science to do so.
Just follow the HB - it's trivial.
Anyway I also second what k.jacker said:
FAT32 isn't suitable for backups
Again: I don't know your exact situation, but for BUs I'd recommend at least a two drives mirror, which with ZFS on FreeBSD is almost as easily created as GPT partition. Don't neccassily needs to create an extra machine for that, but just the pool. While for a two drives mirror you need to attach both drives at the same time, of course.
 
Thank you all for your input.

I think I know the answer, but I'll ask anyway...

If I'm using the entire drive for storage only, I don't need to create a GPT partition table at all and I don't need to create any partitions, correct? In other words, I can simply:

sudo mkexfatfs -n volume_name /dev/daX

and call it good, right?

Ed
Windows will not like that.
 
I'm looking for a recommendation for a filesystem that is "universal" in the sense that I can take the USB drives and routinely move and connect them to FreeBSD, Debian and Microsoft Windows PCs.
Do you think of sharing the data on the disks between OSes? If not, I would split up your disks into two partitions at least and dedicate one to FreeBSD with UFS or ZFS, while maybe use NTFS on the other partition. To be honest, I would never ever connect an important backup to another OS. In my opinion, the latest backup should stay offline anyway, until the next backup is written to another disk/pool. So if you have ony two 8GB harddisks, I wouldn't make them a ZFS mirror, even if that makes up for a ecure backup target. Depending on how many data you have to backup, you may consider creating pools with a single harddrive/vdev and set copies=2 property. That won't protect your pool from harddisk failure, but gives at least redundancy to the data in the pool. Additionally you can create ZFS snapshots of your backup pool and then run rsync to update your backups. That's what I do (automated), but I am a bit paranoid with 4 ZFS mirror pools for backups and one UFS backup target, just in case 😅 And I wouldn't recommend using the entire disk for ZFS. Rather split up in partitions and dedicate those to any OS native filesystem in best case. To share data between OSes, I'd rather set up a nfs or samba share (or use a thumb drive with FAT32 😆).
 
Do you actually need a full file system? With all the crazy complex abilities that POSIX gives you, like hard links, soft links, complex directory structures, attributes, atomic rename, changing files after writing them, deleting of files out of the middle of directories?

If no, here is a suggestion: Use the simplest and most common file system that exists, some variant of FAT. Then store your data in the format of a common archive, for which software exists for all platforms, for example tar or zip. In a nutshell, I'm saying that the USB stick contains just one (or a few) large files, and the complexity of the structure is hidden inside archive software, which is written once.
 
I would never ever connect an important backup to another OS
How much is DVD-ROM or rather DVD-RAM these days? [EDIT: 33 cents for ROM i.e 4.3GB....on par with SSD costs plus significant overhead]
That is how we used to roll backups/archives. You can safely read it everywhere if the potentially hostile system has no DVD-burner.
 
I shudder at the thought of using FAT on 8TB drives.
There are ext[2..4] drivers even for Windows, or maybe use NTFS, ... What are the systems needing to read/write these backups? If it is only systems that can use ZFS, by all means, do that. It will give you also the benefit of checksums, so in case of a restore, you get an early warning that the files to be unpacked are defective.

What other operating systems are we talking about?
 
Why shudder? For flash drive, exFAT is optimal choice. I assume you will use is it for archives or transfer of small amount of files, not for intensive work (like working with AutoCAD using file on flash).
 
The cluster size is better than FAT in exFAT, but when you have smaller files it's simply a waste. And AFAIR, no FAT system has any way of error detection. I simply like that in ZFS. It was argues above that exFAT is slower due to using fuse, and I was referencing FAT here. Plain FAT is something I try to avoid where possible. I'd rather do "tar" directly on the device then.
 
I am not sure you can see speed difference because of fuse for flash drive with write speed like 20-40MB/s. Error detection... if you are copying archive, there is checksum. I hope you have used USB drive with FAT32 format in the last 10+ years. How many times you have detected that something is wrong with your files? Are all your PCs with ECC RAM?
 
I hope you have used USB drive with FAT32 format in the last 10+ years. How many times you have detected that something is wrong with your files? Are all your PCs with ECC RAM?
Yes, I did use them and I had sticks fail in silent or spectacular ways. I want a warning that the files are not OK.

The controller chip is not sufficient, IMHO. I had drives fail while their SMART was dead certain all was well, only "zfs scrub" told me that data was bad and had been reconstructed. Also, data protecting only the content of an archive does not cover the meta data for the file system. And what has ECC Ram in my house is my long-running workstation which has the 10 disc storage pool for backups and documents. Flipping bits in the write cache is what nobody wants.
 
Back
Top