Full Backup solution including Restore

vukodlak

vukodlak

I have my FreeBSD box setup perfectly. When I invariably mess something up I want to be able to restore everything to the perfect setup.
How do I go about doing this?

There must be a way to make a full backup into 1 file and then I can just upload that to gdrive or whatever and always be able to return to this point?
My drive is setup with ZFS btw.
The information I found in google and here is just so conflicting that I am confused to what will actually work...


Thanks!
 
I don't believe there is a simple solution for this and my observation is that everyone has their own solution (solutions +1). Perhaps that is a business opportunity. That said, I think I have a pretty good solution that I'm happy with.

That said, my 'solution' to this problem is:
1. freebsd-installer with repositories for each 'system' I want to build: https://github.com/walterjwhite/freebsd-installer, https://github.com/walterjwhite/freebsd (very old, stripped down example)
2. configuration (https://github.com/walterjwhite/configuration)
3. system-maintenance (https://github.com/walterjwhite/system-maintenance), essentially updates my base system and jails in a safe and predictable manner.

I routinely rebuild my systems from the ground up so everything is backed up, if it isn't, then it is a 'transient' setting in which I should have done a better job backing up in the first place. I'm in the process of virtualizing everything (including my workstation). I plan to run my workstation in a jail and thus my base system is just the kernel and a very stripped down OS with just the basic tools.

Another common approach is to make backups of your ZFS volumes so you can quickly restore from that. That would also be a good approach since you have ZFS, you can leverage ZFS snapshot, and Boot Environments if you want at the system level.

For my media, I use ZFS snapshots and I replicate those to 3 separate drives offline / off-site.

Whatever abstraction works for you will require some effort.
 
Thank You for your answer!
Now I am wondering what they do in a commercial environment? Just use a Raid configuration?
 
You can just save the result of `zfs send` to a file and put that somewhere.

There is no requirement that zfs send streams have to go into a target zfs via zfs receive.
 
Yeah.

The drawback is of course that you cannot cherry-pick individual files to restore from such an archive. It is full restore only unless you have a spare disk.
 
jbo said:
Example from the docs:
zfs send pool/fs@snap | gzip > backupfile.gz
Click to expand...
Thanks for that. I was getting ready to recommend tar.archives I use with UFS and saw mentioned ZFS and backed out.
My thought was pipe zfs to tar.achieve.

But then it became really mentally twisted when I considered compressed datasets.
You wouldn't want to tar/gzip compress a ZFS compressed dataset right?
Does zfs send with pipe remove compression? I know double coompression possible and would save space but very slow.
 
The more I think about this the more I am figuring that there has to be a way to do this. I have HOURS invested in this box. The thought of having to do it all over again is not acceptable.
I've backed up every important file I can think of... rc.conf loader.conf etc etc. Maybe I should just use a ghost program and do a drive image save?
 
It does appear that ZFS send and receive is a way to do this.

Can you do a fresh bsd install and then zfs receive a snapshot back to the new system and be right back where you left off when you did the snapshot?
 
vukodlak said:
It does appear that ZFS send and receive is a way to do this.

Can you do a fresh bsd install and then zfs receive a snapshot back to the new system and be right back where you left off when you did the snapshot?
Click to expand...
You'll still have a bit of work to do. Here's a rough outline of what's needed

Boot from a FreeBSD installer image for the same version which created the backup. Don't bother to run the installer, just select the Live CD option
If necessary use gpart to partition your disk
Create a new zpool on your disk. You might need to destroy any existing pool on the disk
Run zfs receive to restore your data to the new pool
If necessary install FreeBSD bootcode
Set the default bootable dataset with zpool set bootfs=....
Reboot

Things could likely go wrong when you first try this until you've got things sorted so do your research first and do not try restoring over a good working system. You really need to experiment with a spare drive and ideally with your working drive disconnected.
 
That is a way to do it. Drawing back on my example, all touchpoints are backed up, you can repeat that process as outlined above. RAID is not a backup, it is just redundancy.
 
A completely different approach would be to virtualize the thing and run FreeBSD as an image.
Probably does not cost much of the performance anyway, depending on the workload of course.

I really love my virtualized environment, just do snapshots or copy images. Takes 2 minutes to restore whatever I messed up.
 
vukodlak said:
I have my FreeBSD box setup perfectly. When I invariably mess something up I want to be able to restore everything to the perfect setup.
How do I go about doing this?

There must be a way to make a full backup into 1 file and then I can just upload that to gdrive or whatever and always be able to return to this point?
My drive is setup with ZFS btw.
The information I found in google and here is just so conflicting that I am confused to what will actually work...


Thanks!
Click to expand...
The system I export with # bectl export default | xz -9 > /mnt/bkp/<date '+%y-%m-%d'>/"zroot-$(freebsd-version).xz".

And /home I made a full snapshot backup every month and a diferencial backup every week.

Bash: 
{beastie} Ψ FreeBaSeD-T430 Ψ /home/beastie
   Ψ ls --tree /mnt/bkp
 bkp
├──  12023-06-30
│   ├──  zhome.xz
│   └──  zroot-13.2-RELEASE.xz
├──  12023-07-10
│   └──  zroot-13.2-RELEASE-p1.xz
├──  12023-07-17
│   ├──  zhome.xz
│   └──  zroot-13.2-RELEASE-p1.xz
├──  12023-07-24
│   ├──  zhome.xz
│   └──  zroot-13.2-RELEASE-p1.xz
├──  12023-07-31
│   ├──  zhome.xz
│   └──  zroot-13.2-RELEASE-p1.xz
└──  based
    └──  zhome-based.xz
{beastie} Ψ FreeBaSeD-T430 Ψ /home/beastie
   Ψ du -a /mnt/bkp 
 67G    /mnt/bkp/based/zhome-based.xz
 67G    /mnt/bkp/based
2.7G    /mnt/bkp/12023-07-24/zhome.xz
 28G    /mnt/bkp/12023-07-24/zroot-13.2-RELEASE-p1.xz
 31G    /mnt/bkp/12023-07-24
681M    /mnt/bkp/12023-06-30/zhome.xz
 26G    /mnt/bkp/12023-06-30/zroot-13.2-RELEASE.xz
 26G    /mnt/bkp/12023-06-30
 35G    /mnt/bkp/12023-07-10/zroot-13.2-RELEASE-p1.xz
 35G    /mnt/bkp/12023-07-10
 31G    /mnt/bkp/12023-07-31/zroot-13.2-RELEASE-p1.xz
3.8G    /mnt/bkp/12023-07-31/zhome.xz
 34G    /mnt/bkp/12023-07-31
 28G    /mnt/bkp/12023-07-17/zroot-13.2-RELEASE-p1.xz
2.4G    /mnt/bkp/12023-07-17/zhome.xz
 30G    /mnt/bkp/12023-07-17
225G    /mnt/bkp
225G    total
{beastie} Ψ FreeBaSeD-T430 Ψ /home/beastie
   Ψ zfs list -t snapshot
NAME                                             USED  AVAIL     REFER  MOUNTPOINT
tank1/zhome@base-FreeBaSeD-T430_2023-06-30      1.12G      -     77.5G  -
tank1/zhome@FreeBaSeD-T430_2023-07-10           1.11G      -     77.7G  -
tank1/zhome@FreeBaSeD-T430_2023-07-17           1.07G      -     77.7G  -
tank1/zhome@FreeBaSeD-T430_2023-07-24-01:00:39  1.15G      -     77.8G  -
tank1/zhome@FreeBaSeD-T430_2023-07-31-02:10:05  1.21G      -     77.9G  -
zroot/ROOT/default@2023-07-17-12:36:49-0        14.0G      -     57.0G  -
zroot/ROOT/default@2023-07-24-01:13:51-0        12.4G      -     57.2G  -
zroot/ROOT/default@2023-07-31-02:12:03-0         775M      -     57.9G  -
{beastie} Ψ FreeBaSeD-T430 Ψ /home/beastie
   Ψ bectl list -s
BE/Dataset/Snapshot                                Active Mountpoint Space Created

13.2-FreeBaSeD-T430_2023-07-17_124000
  zroot/ROOT/13.2-FreeBaSeD-T430_2023-07-17_124000 -      -          8K    2023-07-17 12:36
    zroot/ROOT/default@2023-07-17-12:36:49-0       -      -          14.0G 2023-07-17 12:36

13.2-FreeBaSeD-T430_2023-07-24-011300
  zroot/ROOT/13.2-FreeBaSeD-T430_2023-07-24-011300 -      -          8K    2023-07-24 01:13
    zroot/ROOT/default@2023-07-24-01:13:51-0       -      -          12.4G 2023-07-24 01:13

13.2-FreeBaSeD-T430_2023-07-31-021100
  zroot/ROOT/13.2-FreeBaSeD-T430_2023-07-31-021100 -      -          8K    2023-07-31 02:12
    zroot/ROOT/default@2023-07-31-02:12:03-0       -      -          775M  2023-07-31 02:12

default
  zroot/ROOT/default                               NR     /          86.0G 2023-06-09 22:02
  default@2023-07-17-12:36:49-0                    -      -          14.0G 2023-07-17 12:36
  default@2023-07-24-01:13:51-0                    -      -          12.4G 2023-07-24 01:13
  default@2023-07-31-02:12:03-0                    -      -          775M  2023-07-31 02:12

The diferencial backup is made like this # zfs send -I tank1/zhome@base-FreeBaSeD-T430_2023-06-30 tank1/zhome@FreeBaSeD-T430_2023-07-31-02:10:05 | xz -9 > /mnt/bkp/<date '+%y-%m-%d'>/zhome.xz.

Then I upload the directory in my extra hd with the weekly backup to my online storage.
 
This is from user vermaden

Its how I run backups on my desktop system. Along with tar for my home directory.
How should a backup of the system be done? (along with applications/data/permissions preserved)

What are the folders/files that are most essential ?

How would such a backup happen if it's a zfs system?

You can just export entire *ZFS Boot Environment* to a file and you have entire system secured.

Backup like this:

Code: 
# beadm list
BE        Active Mountpoint  Space Created
13.1      NR     /           33.4G 2023-01-10 16:55
13.1.safe -      -            2.1G 2023-01-20 17:09

# beadm export 13.1 | xz -9 > 13.1.BE.xz


Restore like this:

Code: 
# xz -c -d 13.1.BE.xz | beadm import 13.1.RESTORE

# beadm list
BE           Active Mountpoint  Space Created
13.1         NR     /           33.4G 2023-01-10 16:55
13.1.safe    -      -            2.1G 2023-01-20 17:09
13.1.RESTORE -      -           31.2G 2023-02-06 18:15

# beadm activate 13.1.RESTORE

When doing 'Bare Metal Recovery' just install same (or newer) FreeBSD version with Auto (ZFS) scheme - does not matter if with GELI enabled or not.

Then do these and reboot(8):



Code: 
# xz -c -d 13.1.BE.xz | beadm import 13.1.RESTORE

# beadm activate 13.1.RESTORE

# reboot

You can also use the same method to 'move' your system to other/new hardware or between VM and a real hardware.

Hope that helps.
Click to expand...


Just keep in mind that boot environments do not include your home directory. Its always best to keep those separate from a compressed boot environment in case you have to to recover individual files.
 
vukodlak said:
When I invariably mess something up I want to be able to restore everything to the perfect setup.
Click to expand...
My take is - use ZFS Boot Environments.

Once your system is up and running create new BE and place it on other system/server/disk/cloud/...

If you mess/broke your system or you need to restore it from scratch - then do new fresh FreeBSD install with ZFS on that host - then transport that BE from that other system/server/disk/cloud/... to you new system and then do beadm import of that BE.

Then activate the BE and reboot.

Procedure complete.
 
cracauer@ said:
You can just save the result of `zfs send` to a file and put that somewhere.
Click to expand...
jbo said:
I never though of this but yeah... one can just pipe it to "anywhere".
Example from the docs:
Code: 
zfs send pool/fs@snap | gzip > backupfile.gz
Click to expand...
cracauer@ said:
Yeah.

The drawback is of course that you cannot cherry-pick individual files to restore from such an archive. It is full restore only unless you have a spare disk.
Click to expand...

All UFS here forever so maybe dumb question, but can dump(8) and restore(8) be used on ZFS filesystems?

Neither those manuals nor 20.9.1. File System Backups indicate whether or not that might work?

If so, then cherry-picked file restores would be possible, but it's unclear to me if ZFS can be considered to be a 'filesystem' as such?
 
is this really as simple as it seems with beadm?
everything seemed to have worked fine in the creation part. Ive not tested the restore part yet but it looks plausible.


CREATE:
beadm create bsdboot
------------------------------
beadm list -a
BE/Dataset/Snapshot Active Mountpoint Space Created

default
zbsd/ROOT/default NR / 5.0G 2023-07-24 17:12

13.2-RELEASE_2023-07-24_172244
zbsd/ROOT/13.2-RELEASE_2023-07-24_172244 - - 8.0K 2023-07-24 17:22
default@2023-07-24-17:22:44-0 - - 40.9M 2023-07-24 17:22

bsdboot
zbsd/ROOT/bsdboot - - 8.0K 2023-08-03 10:36
default@2023-08-03-10:36:01 - - 596.0K 2023-08-03 10:36
--------------------------------

VERIFY:
mount -t zfs zbsd/ROOT/bsdboot /mnt/snapshot
ls /mnt/snapshot shows me all of my files are there.

RESTORE:
Send to 2nd drive ada1/jenna for safekeeping....
beadm send bsdboot > /mnt/jenna/bsdboot_backup.be

bring back to clean drive in freebsd live cd....
beadm receive -e new_be_name < /mnt/jenna/bsdboot_backup.be
Click to expand...
 
jdakhayman said:
Just keep in mind that boot environments do not include your home directory. Its always best to keep those separate from a compressed boot environment in case you have to to recover individual files.
Click to expand...
When i ran my test and mounted the boot environment file my home dirs and data were all there.. are you sure on this?
 
vukodlak said:
When i ran my test and mounted the boot environment file my home dirs and data were all there.. are you sure on this?
Click to expand...
Keep in mind that if you simply "restore" a boot environment on the same machine you made it from, regardless from where it came from (in this instance a compressed file) the dataset that the home directory's are in, are still in the zfs pool if we are talking about the same machine without a reinstall. Therefore, it will show up.

But lets say you had a disk failure, and needed to to restore to a new disk. Once you reinstall FreeBSD from image, then import the pool from the file that you exported, and saved to a different location, the home directory will be empty. This is because boot environments do not include that dataset. This is due because 1). your export file could become huge. 2). you would have to restore the entire boot environment just to get at a single file in your home directory, etc.

In the situation that is being discussed in this thread, You are trying to save the system state, as it were, so that is all boot environments do (base system and pkgs that have been installed).

It seems as the configuration that you are alluding to is also possibly a desktop environment and you want to preserve your customizations? These configurations typically live in your home folder and typically configurations to the base system rarely live here, and therefore are not considered for the boot environment, either.

This is why you should also have a separate backup of your home directory. Use tar and .xz to compress and archive the home directory(ies) so they can then be placed after the system has been "restored". Or utilize zfs to send the home directories to a file just a cracauer@ suggested. But then again you can't just grab a single file out of it.

If you have a spare machine or a system powerful enough to run virtualization, test run this so you can experience first hand what it will do.

Learning to backup a system and your data is very important. Many people stop there, and then struggle to restore it. Learn how to backup, but also learn how to restore it as well. Nothing worse learning to do it under the gun.

jda
 
ahhh ok. I get it now.
no biggie to keep a backup of the home directory and copy it over after I am done. Just tar it up every so often.

Getting the system back up as it was with all the firewall, routing, jails, bhyve info was the main goal.
Just for clarification, the beadm snapshot will preserve all of those things?


Thanks for pointing this out!
 
vukodlak said:
is this really as simple as it seems with beadm?
everything seemed to have worked fine in the creation part. Ive not tested the restore part yet but it looks plausible.
Click to expand...

Code: 
beadm send bsdboot > /mnt/jenna/bsdboot_backup.be
(...)
beadm receive -e new_be_name < /mnt/jenna/bsdboot_backup.be

There are no beadm send and beadm receive commands.

There are beadm export and beadm import commands :)
 
smithi said:
All UFS here forever so maybe dumb question, but can dump(8) and restore(8) be used on ZFS filesystems?

Neither those manuals nor 20.9.1. File System Backups indicate whether or not that might work?

If so, then cherry-picked file restores would be possible, but it's unclear to me if ZFS can be considered to be a 'filesystem' as such?
Click to expand...

Hidden level warning :)

You can also have Boot Environments on UFS filesystems:

- https://vermaden.wordpress.com/2021/04/02/ufs-boot-environments/
 
vermaden said:
There are no beadm send and beadm receive commands.

There are beadm export and beadm import commands :)
Click to expand...
Thanks man :)
this is an issue with FreeBSD being around for such a long time. Doing searches for stuff is rough due to all the old information out there.
One day I will learn to just stick with the Handbook....i can hope anyways heh
 
You must log in or register to reply here.
Back
Top