Questions about backup with dump

[big_girl]

Hi,

I just installed 11REL successfully using this how-to and have done all the configurations and ports/packages installations I want.

Now, I'd like to back it up and use it for another install to different-sized disks (via the same how-to) which will all be used on the same system hardware, but I have some really basic questions about how to use dump().

I want to make sure I get everything with config data including e.g. /boot and /etc. I think this comes down to whether to dump the whole disk or individual slices and what exactly should be backed up to save all config data (e.g. /etc/rc.conf and /boot/loader.conf) and installed ports/packages.

From what I've read it seems dump() does not automatically copy everything (?). Anything I want to avoid dumping I went through the filesystem and set chflags nodump, such as /usr/ports and /var (any advice on what to exclude is much obliged). I'm also confused about dumping the slices I have here (as opposed to partitions, which is what I've seen done in examples).

Here's my disk layout info-

df -h

Filesystem             Size    Used   Avail Capacity  Mounted on
/dev/label/eee.elia    744M    183M    501M    27%    /
devfs                  1.0K    1.0K      0B   100%    /dev
/dev/label/eee.elid    1.9G    271M    1.5G    15%    /var
/dev/label/eee.elie    992M    8.1M    904M     1%    /tmp
/dev/label/eee.elif     24G    5.6G     17G    25%    /usr
/dev/label/eee.elig     35G     73M     32G     0%    /home
procfs                 4.0K    4.0K      0B   100%    /proc

gpart show

=>       63  145226049  ada0  MBR  (69G)
         63          1        - free -  (512B)
         64    1572864     1  freebsd  [active]  (768M)
    1572928  143653176     2  freebsd  (68G)
  145226104          8        - free -  (4.0K)

=>      0  1572864  ada0s1  BSD  (768M)
        0  1572856       1  freebsd-ufs  (768M)
  1572856        8          - free -  (4.0K)

=>       0  17956646  label/eee.eli  BSD  (68G)
         0    196608              1  freebsd-ufs  (768M)
    196608   1048576              2  freebsd-swap  (4.0G)
   1245184    524288              4  freebsd-ufs  (2.0G)
   1769472    262144              5  freebsd-ufs  (1.0G)
   2031616   6553600              6  freebsd-ufs  (25G)
   8585216   9371430              7  freebsd-ufs  (36G)

mount

/dev/label/eee.elia on / (ufs, local, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
/dev/label/eee.elid on /var (ufs, local, journaled soft-updates)
/dev/label/eee.elie on /tmp (ufs, local, noatime, journaled soft-updates)
/dev/label/eee.elif on /usr (ufs, local, journaled soft-updates)
/dev/label/eee.elig on /home (ufs, local, journaled soft-updates)
procfs on /proc (procfs, local)

It seems the command I want is something like-
dump -0Lauf - <device> | pigz > backup_file.tar.gz


Thanks for the help,
-bg

 

[The.Silicon.Projects]

DUMP is very very but incredibly slow.
You can do incremental backup, this is one of the main interest and you can compress the archive.

There is another issue... dump refuses to restore image if the partition of the target is smaller than the partition of the original source (even if in fact.... most of space is not used), and in case of incremental restore dump requires a lot of temporary space on the target, so the size of the target must be significantly higher or you should mount a special temporary unit under the temporary directory created in the target to process the incremental snapshot.... so very very complicated.

For these reasons I do not recommend to use dump in your case.
In fact if you simply want to duplicate your system there is a faster way

Just install and use sysutils/cpdup

Read carefully the man page. Create the .cpignore file where you should include the exclusion as .sujournal if applicable etc..
Indeed you should not export a file like .sujournal to another partition, this file will be created automatically on the target as it is exclusively attached to a filesystem.

One generally use cpdup like that

cpdup -x -i0 -s0 pathTOsource pathTOtarget

By default CPDUP doesn't cross mounts points, this is a great, useful and necessary security.
As target will be likely mounted in /mnt, this will prevent a "circular reference" trying to copy target on itself.

So repeat the same process for each mount point. You see that you have not to worry about the size of the target, just verifying there is enough space.

In a canonical way, if you CPDUP a live system you should make a snapshot (assuming you use UFS, for ZFS I know there is also a snapshot system but I can't say more I don't use ZFS)

Look to the manual for mksnap_ffs, but in fact for common desktop system, it is usually not necessary, I can confirm, I made that many time.

This is more critical for server with a lot of "live" data exchanges, freezing them in a snapshot can be required to maintain consistency, but anyway for information and only for those who use UFS filesystem, here is some ideas on the process to follow.

I create a snap of the root partition under the name freeze_ROOT, a .snap directory must be created before in root directory. Operation should be repeated for any other partition.

Also note before that if you have activated UFS Journaling (SU+J), snapshot is not available, but is available if you use GJournal rather than UFS Soft update and Journaling.

mksnap_ffs / /.snap/freeze_ROOT
mdconfig -a -t vnode -o readonly -f /.snap/freeze_ROOT -u 1

This will create a md1 device, just mount /dev/md1 as a source for the cpdup duplication.

Once finished dismount md1, and you can delete the device and the snapshot

mdconfig -d -u 1
rm -f /.snap/freeze_ROOT

So one should include that in a shell script, this would be more comfortable.

 

[Beastie]

dump(8) works on filesystems. So the "device" should be a partition, not a slice.

Dump /, /var, /usr and /home, and you should have everything worth keeping.

/var is necessary. Third-party software installation for instance is registered there. If you restore /usr without /var you won't be able to manage them (list, remove, etc.)

If you have the system's source code installed in /usr/src you may wish to exclude it.

All configuration files are in /boot, /etc (both dumped with /) and /usr/local/etc (dumped with /usr).

 

[SirDice]

From experience I can tell you that a fresh install is, in general, faster than trying to restore an existing system. A restore can take hours while I can do a full re-install in about 20 minutes. I don't backup packages since everything comes from my own repositories, and again, it's faster to just install them than restoring a backup. Configuration of my machines is done through Puppet so I don't need to backup configuration files either. Really the only thing I actually backup is data.

This works well if you have a lot of machines to maintain. For a single system I would just make a list of the installed packages (so you know what's installed) and backup the necessary configuration files. That will be enough to rebuild the machine from scratch. All that's really important is the actual data (home directories, database contents, etc). Those you should backup because there's no other way to restore them in case things go bad.

 

[big_girl]

From experience I can tell you that a fresh install is, in general, faster than trying to restore an existing system. A restore can take hours while I can do a full re-install in about 20 minutes. I don't backup packages since everything comes from my own repositories, and again, it's faster to just install them than restoring a backup. Configuration of my machines is done through Puppet so I don't need to backup configuration files either. Really the only thing I actually backup is data.

This works well if you have a lot of machines to maintain. For a single system I would just make a list of the installed packages (so you know what's installed) and backup the necessary configuration files. That will be enough to rebuild the machine from scratch. All that's really important is the actual data (home directories, database contents, etc). Those you should backup because there's no other way to restore them in case things go bad.

All good, logical points considering this is a desktop not a headless server. This is the first time I've made a whole system backup of a desktop.

In this case, there were issues which make clean installs inefficient-

1) installed ports that take a long time to build on my older FreeBSD box
2) that older box is running older hard drives which are unreliable (same box, multiple system disks)
3) configuration files are identical across the different system disks (granted you point out an alternative)
4) multiple system installs using the same base install

FYI I ended up taking your advice from an earlier post to use tar() for these backups. Seems like the most efficient route.

EDIT: thinking more about this I think the most efficient way here is actually to just build the ports into packages, put creation of all config files into one script, and do clean installs each time.

-bg

 

[wblock@]

There is another issue... dump refuses to restore image if the partition of the target is smaller than the partition of the original source

I'm not sure where you got that. dump(8) is a file-level backup. It does not do the restore, that's a separate restore(8) program. Which restores just fine to partitions smaller than the original, at least if they are big enough to hold the amount of data.
Are we talking about the same program?

FYI I ended up taking your advice from an earlier post to use tar() for these backups.

dump(8) is the only thing that fully understands UFS filesystems. tar(1) does not, although it works. Mostly.
I've routinely used dump(8) and restore(8) to copy systems, and it works fine. It can be slow if you don't use the right parameters. I have documented this:
Backup Options For FreeBSD.