Howto: restoring full system after software failure

D

dalpets

If I cause a disastrous software error and my system becomes unbootable how do I boot to a backup. Is a bootable rescue disk involved?

Thanks for your help.
 
Most important is to keep backups of your valuable data and FreeBSD configuration files.
It's obvious you haven't looked at the FreeBSD Handbook yet, have you?
It should answer a lot of your questions, or at least will help you to be more specific when asking.
 
A rescue disk is one way. Another way is to just reinstall. It depends on what data you have on the system, where else the data is, how well it is segregated, and what the purpose of the system is.

Let me give you two examples. One, you are only using FreeBSD as a desktop, fundamentally run web browsers and similar tools. What little persistent data you have on the machine is also stored elsewhere (for example using a cloud account). You have not invested much time into tuning and tweaking your installation, and run a stock install. In that case, if something goes wrong, just reinstall from scratch. This is sort of the model of "stateless computing", quite popular with virtual servers today (kubernetes, docker and all that). A different example: You run FreeBSD as a hobby, and you greatly enjoy configuring it: screen background exactly the right color, all the icons arranged by some complex guiding principle, the ethernet ports have fancy names and comments on them, and the file system structure and mount point is so artful, it could get an Oscar. In that case, all the investment of time is in the configuration, and reinstalling from scratch would be disastrous. If something breaks, get a rescue disk, and try to repair it. Third example: Your machine is a data server (mostly a file system), and the important (valuable) data is on a separate file system on a separate device. If something goes wrong, disconnected the valuable disk, and then do whatever you want to get the base system back up and running.
 
For our situation a reinstall is the quickest way to get it back up and running again. But we use Puppet to configure things, so the entire configuration is automated. Once a role has been assigned to a machine Puppet takes care of installing and configuring everything. Takes about 20 minutes to re-do a server from scratch. I think our setup sits somewhere halfway between throwaway "stateless computing" and traditional server management. It took me a bit of time to learn how to use Puppet properly but it made maintaining everything so much easier in the long run.
 
Not sure this is pertinent but as a FreeBSD desktop user, I backup my user's /home to my NAS, plus all pertinent OS configuration files. If something goes bad, which has never happened, I just reinstall. After that, I do the usual OS updates, get my configs pulled over from my NAS and restore my user data.
 
dalpets said:
If I cause a disastrous software error and my system becomes unbootable how do I boot to a backup. Is a bootable rescue disk involved?
Click to expand...

For the systems I run for my personal use, I would do a re-install.

Each night a backup is performed for each host, and the resulting backup file is moved off the host to a central backup/archiving server. The backup includes the needed config files and all the important data on the host.

Using that backup, I can do a full reinstall of a host in one to two hours, depending upon the host. I have written a few scripts to assist me in the reinstalls, which has helped reduce the time significantly.

For my needs, this level of recovery is quite appropriate.
 
I use FreeBSD only at a personal level and that's what I do. I have my installs mostly scripted and part of that scripting pulls data and config files from a backup location. I freely experiment knowing that I can reload the machine pretty fast.

It does take an amount of time and testing to write the scripts, but I think it comes out in the wash. At the least it saves me from taking a lot of notes or remembering things I know I'll never remember. Not that I blow up my system all the time, but it's nice to know it's not a big deal if I break it.

Scripting installs is something you can actually do with FreeBSD. Other operating systems are not nearly that flexible or as amenable to installation scripts.
 
I use USB sticks for backup, populate several laptops off the same ones when I rebuild the system and have a different folder for each laptop I copy off to every machine. I keep a copy of /etc to work from to save time editing new system files.

I've never had a USB stick fail and don't back up my files that often as a home user, but should one of my laptop HDD give up the Ghost in the Machine the most I'll lose is a few recent documents or images at worst. That's the only time I've had a catastrophe I couldn't recover from.

With compling ports and configuration of different programs and styles it usually takes about 24 hours to complete depending on machine specs.
 
If you haven't put much time into configuration then it's probably quicker and easier to re-install. However, booting from a live disk can allow you to have a look at, and possibly fix the issue. It could be simple. Besides that's a way to learn stuff. :)
 
I use ZFS and keep a complete remote backup using snapshots every 15 minutes, sysutils/zrepl. If something went wrong[1] I just reinstall the OS and then I do restore the backups/snapshots. The only extra work is the need to re-create a few "custom" ZFS file systems before bringing the snapshots back.

[1] just happened once when a disk failed.
 
You must log in or register to reply here.
Back
Top