Backup Solutions

I've read various threads about backing up, but I was just wondering what software seasoned FreeBSD users use for HDD backups. I've read about Bacula, Clonezilla, dd(1), and dump(8)/restore(8).

I know some of them won't work for my environment (live with zfs(8) and geli(8) slices). Thanks for anything shared.
 
There are a lot of solutions depending on your needs, you can just use rsync if you want to periodically backup just a few files/directories. There's more canned solutions like BackupPC, Bacula and Amanda, each with varying complexity.

For stuff I don't care much about (media directory) I just have rsync in a cronjob that keeps two PCs (GELI/ZFS) in sync. For my servers I run BackupPC, but it doesn't scale that well and I am considering other the other options I mentioned.
 
I'm not sure I qualify as a seasoned FreeBSD user and maybe this isn't really "HDD backup", but file synchronization/backup.

I use a combination of devel/fossil and net/unison-nox11. For files that I need version control and history I use fossil. For files I just want synchronized between my laptop and desktop (and a third system as the backup) I use unison, which can synchronize files in both directions.

http://www.fossil-scm.org/index.html/doc/tip/www/index.wiki
http://www.cis.upenn.edu/~bcpierce/unison/
 
It depends on the system and data to be backed up. restore(8) are discussed in Backup Options For FreeBSD, along with some mention of dd(1), rsync(1), and tar(1).

sysutils/rsnapshot can duplicate data to another system.

I've heard http://www.tarsnap.com is pretty popular.(8) manual page" href="https://man.freebsd.org/cgi/man.cgi?query=dump/man] and restore(8) are discussed in Backup Options For FreeBSD, along with some mention of dd(1), rsync(1), and tar(1).

sysutils/rsnapshot can duplicate data to another system.

I've heard http://www.tarsnap.com is pretty popular.&sektion=8&manpath=freebsd-release-ports">dump/man] and restore(8) are discussed in Backup Options For FreeBSD, along with some mention of dd(1), rsync(1), and tar(1).

sysutils/rsnapshot can duplicate data to another system.

I've heard http://www.tarsnap.com is pretty popular.(8)
 
I would also say I wouldn't qualify as a "seasoned user" of FreeBSD, but regardless, you might want to check out duplicity. I haven't had much personal experience with it yet, but it looks like a pretty nice tool for backing up data. There is also a GUI front-end to it which I believe is called Deja Dup, but I am not aware of a FreeBSD port of it.
 
I'm not sure I qualify as a seasoned FreeBSD user and maybe this isn't really "HDD backup", but file synchronization/backup.

I use a combination of devel/fossil and net/unison-nox11. For files that I need version control and history I use fossil. For files I just want synchronized between my laptop and desktop (and a third system as the backup) I use unison, which can synchronize files in both directions.

http://www.fossil-scm.org/index.html/doc/tip/www/index.wiki
http://www.cis.upenn.edu/~bcpierce/unison/
Which version of Fossil do you use with your FreeBSD installs?

Ken Gordon
 
I have a 12TB HDD in an external USB case. A cron job runs nightly which powers on the drive, mounts the drive file system, zfs send sends a bzipped snapshot to to a file on the drive, the drive file system is unmounted and the drive is powered off. I manually prune the backup drive every couple of months when it approaches 90% full.

In addition a cron job runs hourly to backup all users' email to a remote FreeBSD cloud server.
 
I have a 12TB HDD in an external USB case. A cron job runs nightly which powers on the drive, mounts the drive file system, zfs send sends a bzipped snapshot to to a file on the drive, the drive file system is unmounted and the drive is powered off. I manually prune the backup drive every couple of months when it approaches 90% full.

In addition a cron job runs hourly to backup all users' email to a remote FreeBSD cloud server.
I wonder how could your computer power on and off an external USB drive case?
 
Code:
usbconfig -d ${USB_PORT} power_on && sleep 30 && mount /mnt
...
umount -v /mnt && sleep 60 && usbconfig -d ${USB_PORT} power_off
 
Code:
usbconfig -d ${USB_PORT} power_on && sleep 30 && mount /mnt
...
umount -v /mnt && sleep 60 && usbconfig -d ${USB_PORT} power_off
Oh right, with usbconfig tool. But I guess when you first plug that enclosure into computer, it should be in powered on state, do you keep it like that or you just power it off?
 
I've read various threads about backing up, but I was just wondering what software seasoned FreeBSD users use for HDD backups. I've read about Bacula, Clonezilla, dd(1), and dump(8)/restore(8).

I know some of them won't work for my environment (live with zfs(8) and geli(8) slices). Thanks for anything shared.
I enjoy ZFS since 2008 on FreeBSD - and all my data except cloud backups are kept on ZFS pools. All of them are GELI encrypted.

I sync between my primary storage space and 'backup locations' with rsync(1) over ssh(1) or locally.

I also got some cloud storage and it offers S3 interface - so I use rclone(1) for that - also encrypted with rclone(1) for security.

I really do not like to pay 'monthly' or 'yearly' so I got a 'lifetime' 1TB storage space at cloud provider that supports rclone(1) for sync - it cost me about $120 but it was more then year ago - now it costs about $140 to $160 for 1TB.

- https://cloudwards.net/best-lifetime-cloud-storage/
- https://mashable.com/deals/dec-24-koofr-cloud-storage

Regards,
vermaden
 
My strategy for at least a decade now has been to run ZFS which allows me to do the following:

1. Segregate "read-only but must be online" data (e.g. media files such as purchased music) that do not need to be on the "routine" backups but must not be lost from "read-mostly" data (e.g. older code projects that you might change but infrequently do) and must also not be lost from "read/write routinely" data (e.g. current projects, current emails, etc.)

2. This in turn allows backing up the first to volumes that are (1) mirrored and (2) stored off-site separate from the other data. The read-only backup volumes can then be brought in on some schedule (e.g. annually), updated rapidly since new additions are small (e.g. incremental snapshot), scrubbed to insure they're good and returned to the offsite storage.

3. The "routine" backups then cover only the remainder and the auto-snapshot function works great for this, with said snapshots incrementally updated onto secondary media daily with the backup volumes offline (and thus not mounted, obviously) when not in active use. I have a script that runs through those areas and does this. I use a three-drive set of mirrors for the backup volumes, two of the members are on-site and one is kept offsite; once every week or so I swap the offsite one (e.g. "offline"/"online") and thus resilver.

All of the operating volumes and backups are GELI FDE'd thus if stolen the thief has effectively-blank disks.

In the event of a catastrophic failure of the online and operating volumes (or the HBAs, etc.) I can recover on-site all but the read-only portions immediately, and those are recoverable as soon as I can get to the offsite copies. In the event of physical destruction of the site entirely my exposure is limited to when the last swap of the operational working copy was made. The "online" data, presuming I do not physically lose the site (e.g. fire, tornado, etc.) has a risk window of one day and if I do then the risk window is the swap interval with the offsite location.

I have had to recover "in anger" using this strategy when BOTH mirror members of an operational SSD volume were destroyed by a power supply fault and it was entirely successful.
 
restic paired with rclone is my go to solution. It's fast, reliable and flexible. I actually backup most of my systems twice. Once to a local restic server (though you can back up to another drive as well) and once to onedrive via rclone.
 
Since S3 does not provide a convenient way to have WORM (write once read many) backups, I chose something different. I have done extensive tests on restic vs borgbackup on various datasets and for my use-cases borgbackup is the best solution. I can highly recommend (I am not affiliated with hetzner) the hetzner storagebox from https://www.hetzner.com/storage/storage-box - you get a trimmed down shell with rclone/borg command.
 
Oh right, with usbconfig tool. But I guess when you first plug that enclosure into computer, it should be in powered on state, do you keep it like that or you just power it off?
I shut it down after plugging in for the first time. I also have a startup script which runs to shut it down after a system reboot or poweroff.
 
I don't HDD backup; backing up files is easier
...and quicker.


About Backup Solutions:

“There are two types of people:
those who have suffered a catastrophic loss of data,
and those who will.”


When desaster strikes - and sooner or later it will - its's either no big deal at all. You have sufficient backups to fall back on. There may only the need to download, reinstall, and then copy things bayk. Annoying, yes, but no serious problem, no real damage done, since no data, no time you spent on things you produced and configured, is lost.
Or, if not, it's simply your backup plan ain't sufficient.

Backups by full storage drives and partition clones I did under Windows. I found no better way to backup a Windows system, since with Windows the system, its configs, partition schemes, and what unkown junk else are amalgamated to one ugly, inseparable cast part. Anything but nice.
There is software for that like gparted-live, or clonezilla. But dd does it also, if you just want to backup/restore full drives, and not align/resize partitions.
Pro: They are trivial to do. Nothing to setup, configure, think about... Just simple and easy save everything.
Cons: They save everything. Drives are never full. With full drive BUs (clones) you also backup empty sectors, which is pretty pointless.
Also full drive BUs take way more time as it is practical for regular, not even daily BUs. So they cannot be done frequently enough to be really useful for normal production usage, but as part of larger system's modifications, only. If used as only BUs there is the trap your last BU is very old - too old, and with a restoration you enter the "way back time machine." Months of lost work and data could be the result.

And that's what's most importanat to be backed up: Your work. All and any data you spent time on.
The system itself is "throw away", 'cause it's easy to download and reinstall again. Even quickly done, when your backup-plan covers your system's config files, and a list of your packages installed. pkg prime-list > mypackages.txt produces a textfile containing all names of all packages you installed on your system without version number, so a reinstallation of all packages can be automated easily.
Also the hardware is quickly replaceable. But no manufacturer's guarantee covers your paper you're working on for months, that has its deadline tomorrow.

Under a real OS (😁) in almost all cases it's to backup the files.

That was the easy part.
Now it will become not really complicated, but it needs a bit effort to answer questions for to analyze the own situation, for to develop an own, reasonable backup plan.

First classify what to backup, e.g. by WISP:
  • What? /home (of course!), config files(!): /etc, /usr/local/etc (X11 config), plus data, repositories,... /saved-games, /ex-partner-pix,... temp-files or trash.bin contents(?)
  • Importance and value: Lifes depend on it, important, less important, unimportant, garbage ... How hard is it to get it again. How much effort takes it to reproduce?
  • Size (some kB...several TB)
  • Period frequency: How often to bu which data? monthly, weekly, daily,...every second? [snapshots]) Which data changes how often? You don't need to backup HD movie files (many GB up to TBs) every day, while you better backup your current production directories (several hundred kB up to a few MB) at least daily.
The amount of data to be backupped, the frequency how often the backups shall be done, the available storage capacity of where the backups live, and the bandwidth - how many byte per second can be transferred/written to the backup medium - needs to be evalutated as also where to backup to, which depends on severaö factors:
  • What's reasonable? (Do large files which don't change need to stress a regulary BU routine?)
  • What needs to be safe from what; own idiocy [#1 reason to restore from backups], hardware failure [can happen anytime, even on brand new hardware, #2], burning house, nature catastrophy, burglary, malware, cracker attack...
  • How long need the backups endure (importance and frequency of change)
  • Where to (additional single drive or mirror, NAS, flashdrive (not as BU medium; if even only unimportantjunk), cloudserver (💲), tape, CD/DVD/BD,...) What's available? What's affordable? What suffices best for what?
  • Available storage capacity
  • Available bandwidth (1 kb/s...10 GB/s...?)
  • How (cp, rsync, dump,... tar, compress, encryption,... incremental, non-incremental, self written shell scripts, backula,...verification)
  • Redundancy (1x, 2x, 3x...extra copies)
It's obvious:
There cannot be given a general plan, except "Do backups!" - "Do it! Do it! Do it! NOW!"

Everybody has to develop the own suiting plan based on the analysis of the very own situation.
Setting up a backup plan is not only part of setting up a system like think of which window manager to use, or to install packages, but the most important part.
Check, adapt, improve - upgrade your backup plan regulary is part of system's maintenence, any system's administration, no matter how stable or "safe" the system is.
It's even more important than to check for updates, and upgrades for softwarepackages, and less work when you have a good plan.

Anyway, I want to give a general tip, that seems "Duh! Obvious!" but may be underestimated when desaster striked:
Test the restoration! Really, actually do it!
All backup software, and backup strategies are worthless if you don't know how to get your data back, which ain't so certain to rely on if you never did it. ☝️🤓
When the real event occurs - and it will - you will be anything but cool. So it's better you are at least cool and confident in knowing how to handle your restoration, to not mess things up even more. :cool:

See also:
The FreeBSD HB 20.8 Backup Basics
Backup - Wikipedia
"Backups aren't simple" by Aleksandar Filipovski
 
Last edited:
Back
Top