Backup Solutions

I've read various threads about backing up, but I was just wondering what software seasoned FreeBSD users use for HDD backups. I've read about Bacula, Clonezilla, dd(1), and dump(8)/restore(8).

I know some of them won't work for my environment (live with zfs(8) and geli(8) slices). Thanks for anything shared.
 
There are a lot of solutions depending on your needs, you can just use rsync if you want to periodically backup just a few files/directories. There's more canned solutions like BackupPC, Bacula and Amanda, each with varying complexity.

For stuff I don't care much about (media directory) I just have rsync in a cronjob that keeps two PCs (GELI/ZFS) in sync. For my servers I run BackupPC, but it doesn't scale that well and I am considering other the other options I mentioned.
 
I'm not sure I qualify as a seasoned FreeBSD user and maybe this isn't really "HDD backup", but file synchronization/backup.

I use a combination of devel/fossil and net/unison-nox11. For files that I need version control and history I use fossil. For files I just want synchronized between my laptop and desktop (and a third system as the backup) I use unison, which can synchronize files in both directions.

http://www.fossil-scm.org/index.html/doc/tip/www/index.wiki
http://www.cis.upenn.edu/~bcpierce/unison/
 
It depends on the system and data to be backed up. restore(8) are discussed in Backup Options For FreeBSD, along with some mention of dd(1), rsync(1), and tar(1).

sysutils/rsnapshot can duplicate data to another system.

I've heard http://www.tarsnap.com is pretty popular.(8) manual page" href="https://man.freebsd.org/cgi/man.cgi?query=dump/man] and restore(8) are discussed in Backup Options For FreeBSD, along with some mention of dd(1), rsync(1), and tar(1).

sysutils/rsnapshot can duplicate data to another system.

I've heard http://www.tarsnap.com is pretty popular.&sektion=8&manpath=freebsd-release-ports">dump/man] and restore(8) are discussed in Backup Options For FreeBSD, along with some mention of dd(1), rsync(1), and tar(1).

sysutils/rsnapshot can duplicate data to another system.

I've heard http://www.tarsnap.com is pretty popular.(8)
 
I would also say I wouldn't qualify as a "seasoned user" of FreeBSD, but regardless, you might want to check out duplicity. I haven't had much personal experience with it yet, but it looks like a pretty nice tool for backing up data. There is also a GUI front-end to it which I believe is called Deja Dup, but I am not aware of a FreeBSD port of it.
 
I'm not sure I qualify as a seasoned FreeBSD user and maybe this isn't really "HDD backup", but file synchronization/backup.

I use a combination of devel/fossil and net/unison-nox11. For files that I need version control and history I use fossil. For files I just want synchronized between my laptop and desktop (and a third system as the backup) I use unison, which can synchronize files in both directions.

http://www.fossil-scm.org/index.html/doc/tip/www/index.wiki
http://www.cis.upenn.edu/~bcpierce/unison/
Which version of Fossil do you use with your FreeBSD installs?

Ken Gordon
 
I have a 12TB HDD in an external USB case. A cron job runs nightly which powers on the drive, mounts the drive file system, zfs send sends a bzipped snapshot to to a file on the drive, the drive file system is unmounted and the drive is powered off. I manually prune the backup drive every couple of months when it approaches 90% full.

In addition a cron job runs hourly to backup all users' email to a remote FreeBSD cloud server.
 
I have a 12TB HDD in an external USB case. A cron job runs nightly which powers on the drive, mounts the drive file system, zfs send sends a bzipped snapshot to to a file on the drive, the drive file system is unmounted and the drive is powered off. I manually prune the backup drive every couple of months when it approaches 90% full.

In addition a cron job runs hourly to backup all users' email to a remote FreeBSD cloud server.
I wonder how could your computer power on and off an external USB drive case?
 
Code:
usbconfig -d ${USB_PORT} power_on && sleep 30 && mount /mnt
...
umount -v /mnt && sleep 60 && usbconfig -d ${USB_PORT} power_off
 
Code:
usbconfig -d ${USB_PORT} power_on && sleep 30 && mount /mnt
...
umount -v /mnt && sleep 60 && usbconfig -d ${USB_PORT} power_off
Oh right, with usbconfig tool. But I guess when you first plug that enclosure into computer, it should be in powered on state, do you keep it like that or you just power it off?
 
I've read various threads about backing up, but I was just wondering what software seasoned FreeBSD users use for HDD backups. I've read about Bacula, Clonezilla, dd(1), and dump(8)/restore(8).

I know some of them won't work for my environment (live with zfs(8) and geli(8) slices). Thanks for anything shared.
I enjoy ZFS since 2008 on FreeBSD - and all my data except cloud backups are kept on ZFS pools. All of them are GELI encrypted.

I sync between my primary storage space and 'backup locations' with rsync(1) over ssh(1) or locally.

I also got some cloud storage and it offers S3 interface - so I use rclone(1) for that - also encrypted with rclone(1) for security.

I really do not like to pay 'monthly' or 'yearly' so I got a 'lifetime' 1TB storage space at cloud provider that supports rclone(1) for sync - it cost me about $120 but it was more then year ago - now it costs about $140 to $160 for 1TB.

- https://cloudwards.net/best-lifetime-cloud-storage/
- https://mashable.com/deals/dec-24-koofr-cloud-storage

Regards,
vermaden
 
My strategy for at least a decade now has been to run ZFS which allows me to do the following:

1. Segregate "read-only but must be online" data (e.g. media files such as purchased music) that do not need to be on the "routine" backups but must not be lost from "read-mostly" data (e.g. older code projects that you might change but infrequently do) and must also not be lost from "read/write routinely" data (e.g. current projects, current emails, etc.)

2. This in turn allows backing up the first to volumes that are (1) mirrored and (2) stored off-site separate from the other data. The read-only backup volumes can then be brought in on some schedule (e.g. annually), updated rapidly since new additions are small (e.g. incremental snapshot), scrubbed to insure they're good and returned to the offsite storage.

3. The "routine" backups then cover only the remainder and the auto-snapshot function works great for this, with said snapshots incrementally updated onto secondary media daily with the backup volumes offline (and thus not mounted, obviously) when not in active use. I have a script that runs through those areas and does this. I use a three-drive set of mirrors for the backup volumes, two of the members are on-site and one is kept offsite; once every week or so I swap the offsite one (e.g. "offline"/"online") and thus resilver.

All of the operating volumes and backups are GELI FDE'd thus if stolen the thief has effectively-blank disks.

In the event of a catastrophic failure of the online and operating volumes (or the HBAs, etc.) I can recover on-site all but the read-only portions immediately, and those are recoverable as soon as I can get to the offsite copies. In the event of physical destruction of the site entirely my exposure is limited to when the last swap of the operational working copy was made. The "online" data, presuming I do not physically lose the site (e.g. fire, tornado, etc.) has a risk window of one day and if I do then the risk window is the swap interval with the offsite location.

I have had to recover "in anger" using this strategy when BOTH mirror members of an operational SSD volume were destroyed by a power supply fault and it was entirely successful.
 
restic paired with rclone is my go to solution. It's fast, reliable and flexible. I actually backup most of my systems twice. Once to a local restic server (though you can back up to another drive as well) and once to onedrive via rclone.
 
Since S3 does not provide a convenient way to have WORM (write once read many) backups, I chose something different. I have done extensive tests on restic vs borgbackup on various datasets and for my use-cases borgbackup is the best solution. I can highly recommend (I am not affiliated with hetzner) the hetzner storagebox from https://www.hetzner.com/storage/storage-box - you get a trimmed down shell with rclone/borg command.
 
Oh right, with usbconfig tool. But I guess when you first plug that enclosure into computer, it should be in powered on state, do you keep it like that or you just power it off?
I shut it down after plugging in for the first time. I also have a startup script which runs to shut it down after a system reboot or poweroff.
 
I don't HDD backup; backing up files is easier
...and quicker.


About Backup Solutions:

Full HDD/SSD/partition backups I did under Windows. There is software for that like gparted-live, or clonezilla, but dd does it also, if you just want to backup/restore full drives, and not align/resize partitions.
Pro: From a pure technical point of view they are trivial to set up, easily done, and simply safe everything.
Cons: They safe everything. Since drives are never full with full drive BUs you'd also backup empty sectors, which does not make much sense. Full drive BUs often take (way) more time as it was practical for regular daily safes, so they cannot be done frequently enough to be really useful for normal production usage, but as part of larger system modifications, only. If used as only BUs there is the trap your last BU made is very old, and with a restoration you enter the "way back time machine." Months of lost work could be the result.

And that's what needed to be backed up: Your work. All your time you spent.
The system itself is "throw away", 'cause it's easy to reinstall, even quickly done, when your backup-plan covers your system config. Also the hardware is quickly replaceable. But no manufacturer's guarantee covers your paper you're working on for months, that has its deadline tomorrow.

Under a real OS (😁) in almost all cases it's to backup the files, and filesystems, maybe the partition tables.

That was the easy part.
Now it will not become really complicated, but it needs a bit effort to answer questions for to analyze the own situation, and develop an own backup plan:
  • what (system(?), config files(!), /etc, /home, /data, repos,... /ex-partner-pix, /saved-games,... more important, less important, unimportant, life depends on it...)
  • amount (some kB...several TB)
  • how often (monthly, weekly, daily,...every second?[zfs snapshots]) (which data changes how often?)
  • what's reasonable?
  • what needs to be safe from what (own idiocy [#1 reason to restore from backups!], hardware failure [can happen anytime, even on brand new hardware], burning house, nature catastrophy, burglary, malware,...)
  • how long need the backups endure (again: frequency of change)
  • where to (additional drive, NAS, cloudserver, tape, CD/DVD/BD,...) what's available? what's affordable? what suffices?
  • available storage capacity
  • available bandwidth (1 kb/s...10 GB/s...?)
  • How (cp, rsync, dump,... tar, compress, encryption,... incremental, non-incremental, self written shell scripts, backula,...verification)
  • redundancy (1x, 2x, 3x...)
It's pretty obvious: There cannot be given a general answer but "Do backups!" - "do it! do it! do it!" NOW!
Everybody has to develop her own suiting strategy based on the analysis of the very own situation.
Setting up a backup plan is not only part of setting up a system like think of which window manager to use, or install packages, but the most important part.
Check, adjust and improve your backup plan is part of system's maintenence, like updates, and upgrades (maybe less often needed, but even more important.)

Anyway, I want to give a general tip, that seems "duh! obvious!" but may be underestimated when desaster striked:
Test the restoration! Really, actually do it!
All backup software, and backup strategies are worthless if you don't know how to get your data back. If the real event occurs you will be anything but cool. So it's better you are at least cool and confident in handling your restoration, not to mess things up even more.
:cool:
 
Last edited:
Back
Top