Other Backing up to an ext4 external disk

W

willie

Hoping to get some advice about best practices when writing backups to non-native file systems.

Currently, I have an install of 15.1 on the internal sata ssd disk of my laptop, and have rsnapshot set up to make redundent copies of /home, /etc and /usr/local to a different ufs partition on the same disk. Seems to work fine as I have extracted some files from the backup to repair my email client after a configuration mishap.

I also have a larger 2.5" ssd external disk tethered with a usb adapter, that contains quite a bit of archived data , is intended to be stored offsite and to be also used for back-ups of /home. The partitions on it are ext4, as I have been using linux for a while previous.

Creating a first time rsnapshot to the linux disk took longer than expected, even accounting for the usb slowness. Of course in a perfect world I would have an external backup disk formatted ufs for freebsd. And a different backup disk just for linux installs, but then I would still need to sync /home across 2 different file types.

would dump/restore or rsync be a better choice than rsnapshot?

am I better off writing to ext4 fs volumes with freebsd drivers or looking for a way to write to ufs from linux?
 
In general, I would not use "foreign" file systems (like ext4 on FreeBSD, or UFS on Linux, or NTFS on either) in writing mode. And I wouldn't trust them in reading, but at least reading is not destructive if you mount the file system read-only (no atime updates). Why? Because the foreign file system implementations are not well tested, usually don't share the same source code, and are sometimes written not by understanding how the file system works, but by trying to recreate the artifacts it leaves on disk. The situation is different for ZFS, which shares a common code base between Linux and FreeBSD.

Now, that comment is not very helpful: It tells you what NOT to do, but it doesn't quantify the risk (most of the time, foreign file systems will work). If you want to avoid that risk, you'll have to do some major operation, which might be too much work. You could reformat the external disk in UFS or ZFS and use it natively on FreeBSD, but that is a lot of work. The advantage of using ZFS for that is that you can share it with Linux more safely, as long as you make sure to only use feature flags that are supported by both OSes. Another option would be to actually run Linux to read/write the external disk and keep it ext4. You could set up a little VM on your FreeBSD machine, run a small Linux installation there, use NFS on it to communicate with FreeBSD. Or you could get a tiny computer (Raspberry Pi?) as a Linux server for the disk. But any of that is a lot of work.
 
In general I agree with ralphbsz but I think a question is how/where do you intend on accessing this backed up data?
Data created on a FreeBSD system then accessed on a Linux system?
Data created on FreeBSD only accessed on FreeBSD?
Those questions I think feed into what ralphbsz is saying.

NFS:
in theory NFS is the universal filesystem. It doesn't matter what the native filesystem is, if it's exposed by NFS the FreeBSD, Linux, Windows, MacOS can read/write it.
Heck NFS is a better/easier choice on VirtualBox than the VirtualBox native "shared folders"
 
For non-native file systems, exFAT is better than ext4 (or FAT32 for size less 16GB). They are simple and with lower risk of hidden bugs.
 
It is time to implement native exFAT. Its structure was secret in the past and implementations were from reverse-engineering and "illegal" but now file system structure is open.
 
mer said:
(the) question is how/where do you intend on accessing this backed up data?
Data created on a FreeBSD system then accessed on a Linux system?
Data created on FreeBSD only accessed on FreeBSD?
Click to expand...

I will still keep a linux install around, so yes, I would like to retain the capability of writing to ext4 with Freebsd. That said, I don't anticipate it would be a lot of data, and one possibility I have considered is formatting a UFS partition on the backup drive alongside some ext4 partitions, and copying data as needed, with FreeBSD, from the UFS partition to an ext4 partition. Or possibly mounting the UFS partition ro from linux. (untested)


NFS:
in theory NFS is the universal filesystem It doesn't matter what the native filesystem is, if it's exposed by NFS the FreeBSD, Linux, Windows, MacOS can read/write it.
Click to expand...

Do you know of a way to implement NFS locally, ie without setting up a new machine or network?
 
cracauer@ said:
Which ext4fs do you use - fuse or the kernel one?
Click to expand...
the mount command run from FreeBSD, returns
/dev/da0p4 on /mnt (ext2fs, local)
and when run from linux....
/dev/sdc4 on /mnt/sdc4 type ext4 (rw,relatime,stripe=8191)

(I dont know if thats enough info to answer the Q)

The best filesystem for sharing between OSes these days is ZFS.
Click to expand...

Would I need to set up a ZFS formatted linux install to send to?
I have considered ZFS for my FreeBSD install, and am curious if rsync, rsnapshot or another is practical for copying from ZFS to ext4.
 
ralphbsz said:
You could reformat the external disk in UFS or ZFS and use it natively on FreeBSD, but that is a lot of work. The advantage of using ZFS for that is that you can share it with Linux more safely, as long as you make sure to only use feature flags that are supported by both OSes.
Click to expand...

Reformatting the backup disk to UFS or ZFS may be in the big picture down the road. Right now, I could add a UFS partition to it as mentioned above. Actually I am in the process of consolidating data from many years past and many ext2,3,4 formatted disks to a new ssd USB attached backup disk, and I have the opportunity to partition and format it any way I choose.
My laptop has two bays for a possible ZFS mirror setup with identical 240 GB ssd's, (which is attractive to cover any redundancy needs), and will be possible once I free up some older disks after extracting data I wish to archive on the new backup disk.

One thing I hope to avoid is, once I consolidate my archived data to the backup USB ssd, is having to copy it again in the future with less than stellar results. There have been a few times when I wish I had given more thought to setting up a disk for future use, hence all the questions.
 
willie said:
am I better off writing to ext4 fs volumes with freebsd drivers or looking for a way to write to ufs from linux?
Click to expand...

For FreeBSD - I use "rsync" to backup what I want to save to a USB mounted UFS formatted drive
For Linux - I use "rsync" to backup what I want to save to a USB mounted ext4 formatted drive

My plan is always to restore data directly back to either FreeBSD or Linux directly from where the data came from.

I "could" mix and match "backup data" between the O/S'es -- but that can get confusing (at least to me! :cool: ) and I don't want to distract myself from what I am REALLY trying to do, which is to restore and/or recreate a FreeBSD or Linux system back to "what it was".

If I want to copy data directly from FreeBSD to Linux (or) Linux to FreeBSD -- I use a USB FAT32 formatted drive memory stick. Of course I can also sftp/ssh data freely between Linux and FreeBSD hosts as well. I could (ALSO) do NFS, SAMBA/CIFS, or something similar between FreeBSD and Linux hosts -- but again, it just depends on what I am trying to accomplish.

I have nothing against ZFS for this purpose as well ? But at the moment I am very happy with UFS for my backups, I have always had GREAT success with UFS formatted drives for a lot of purposes.
 
willie said:
the mount command run from FreeBSD, returns
/dev/da0p4 on /mnt (ext2fs, local)
and when run from linux....
/dev/sdc4 on /mnt/sdc4 type ext4 (rw,relatime,stripe=8191)

(I dont know if thats enough info to answer the Q)
Click to expand...
Yes, I can see that you use the in-kernel ext2fs, so that is all the performance you gonna get.

willie said:
Would I need to set up a ZFS formatted linux install to send to?
I have considered ZFS for my FreeBSD install, and am curious if rsync, rsnapshot or another is practical for copying from ZFS to ext4.
Click to expand...

Your Linux install doesn't have to use ZFS for boot or anything.

rsnapshot is no problem with ZFS.
 
CShell said:
If I want to copy data directly from FreeBSD to Linux (or) Linux to FreeBSD -- I use a USB FAT32 formatted drive memory stick.
Click to expand...
Hi Cshell,
In the case where I am migrating from linux to FreeBSD, and data that I want to export into FreeBSD is currently stored on an ext4 partition, would you go so far as to
1. boot into linux in order to copy/write a file from an ext4 partition to a Fat32 partition.
then
2. reboot into FreeBSD in order to copy/write that file from the Fat32 partition to UFS/ZFS.

I understand that writes directly from linux to FreeBSD (or vice versa ) may not be as well supported as read only, but we only need to mount an ext4 partition read only in order to copy/write it to UFS/ZFS
 
willie said:
Kent,
Is rsnapshot or rsync lacking in some ways? I have considered tar instead
Do you create incremental or differential backups with tar?
Click to expand...
tar (tape archiver) is a universally recognized utility that many of the historical "enterprise backup" solutions use on the backside. It does allow for incremental backups as well. It is a complex program with many options that should meet your needs. The downside (if you cant to call it that) is that is was for backing up to tape, so there is no "directory" information stored at the beginning of the archive: just file headers spread throughout the stream...so finding/reading a particular file requires a linear search...The workaround for that is to limit the archive size and use a sequence of archive files for different paths. You can also force block archives to be a chosen maximum size.
 
What's important to use ZFS common between multiple OSes (including different version of the same OS) is limiting features for least-featured one, at least for read-incompatible features.
 
T-Aoki said:
What's important to use ZFS common between multiple OSes (including different version of the same OS) is limiting features for least-featured one, at least for read-incompatible features.
Click to expand...
Very important.

After creating the pool, test import on all desired platforms before populating it.
 
cracauer@ said:
Very important.

After creating the pool, test import on all desired platforms before populating it.
Click to expand...
Maybe the easiest would be to create the pool on least-featured OS.

But unfortunately, "least featured" does NOT always mean "all enabled features are available on all other OSes". So as you mentioned, testing imports on all to-be-shared OSes without writing any data is important to allow disabling the offending features after tests.
 
using ZFS for transportable backups is foolish and I'd chastise my engineers with "bad engineer! no cookie!" for such a path. a low denominator like EXT2 or FAT32 would be preferred, especially when layering backup data in something like tar and limiting archive filesize. KISS, keep it simple, stupid.
 
You must log in or register to reply here.
Back
Top