dump -- I just don't get it

K

kr651129

I read the handbook on using dump and I guess I just don't understand. Can I use dump to take an image of my entire disk incase of failure and restore my hdd to exactly how my pc is functioning from the day of the image?
 
If you are really looking for cloning the complete disk, including all partitions, then dump/restore is perhaps not the best option. It is somewhat slow, and as wblock pointed out, you have to take care for each partition and the boot block separately.

Quite frequently, I use dd(1)() to clone my system disk to another hdd of the same model. This produces an exact clone, including partitions and its labels.

dd if=/dev/ada0 of=/dev/ada1 bs=1m conv=noerror

Here /dev/ada0 is the source disk and /dev/ada1 is the target. On a low profile system (Dual Core Atom, 1.6 Ghz), cloning of a 3 Tbyte disk (2.1 Tbyte used) takes about 12 h at an overall transfer rate of roughly 70 Mbyte/s. Processor load by dd was about 2-3 % during cloning.

Actually, I always submit the dd command from within tmux(1)(), and then detach from the session.

Best regards

Rolf
 
rolfheinrich said:
If you are really looking for cloning the complete disk, including all partitions, then dump/restore is perhaps not the best option. It is somewhat slow, and as wblock pointed out, you have to take care for each partition and the boot block separately.

Quite frequently, I use dd(1)() to clone my system disk to another hdd of the same model. This produces an exact clone, including partitions and its labels.

dd if=/dev/ada0 of=/dev/ada1 bs=1m conv=noerror
Click to expand...

A couple things:

1. dump(8) can be much faster than dd(1), because most disks aren't entirely full or even close. dump(8) copies just the blocks that are used, dd(1) copies *every* block.

2. noerror means "if there is an error during the copy, no big deal, just fill it with zeros and keep going". Do not use noerror unless the copy is from a failing hard drive.
 
wblock@ said:
1. dump(8) can be much faster than dd(1), because most disks aren't entirely full or even close. dump(8) copies just the blocks that are used, dd(1) copies *every* block.
Click to expand...

dump(8)/restore(8) doesn't even remotely come close to a transfer rate of 70 MByte/s. A recent experiment with the same disks gave 15 MByte/s, and at the same time the processor utilization was quite high.

# mount -o async /dev/ada1p3 /mnt; cd /mnt
# dump -C 24 -0af - /dev/ada0p3 | restore -rf -

Code: 
 1926 root        1  58    0  4324K  2256K piperd  1   0:06 18.90% restore
 1927 root        1  57    0 28004K  7008K sbwait  3   0:05 17.48% dump
 1928 root        1  49    0 27876K 26044K physrd  3   0:02  8.50% dump
 1929 root        1  49    0 27876K 26044K physrd  1   0:02  8.40% dump
 1930 root        1  49    0 27876K 26044K physrd  1   0:02  8.15% dump

This means, that cloning my 3 Tb disk would take 12*70/15*2.1/2.7 = 43h 33min, and during this time the system would be also quite in-responsive, because of the load that dump/restore imposes on to the CPUs.

So in my case, dump/restore would be superior to dd only if the disk to be cloned would be only about 21 % full.

wblock@ said:
2. noerror means "if there is an error during the copy, no big deal, just fill it with zeros and keep going". Do not use noerror unless the copy is from a failing hard drive.
Click to expand...

The relevant section of dd(1)() tells:

Code: 
noerror  Do not stop processing on an input error.  When an
         input error occurs, a diagnostic message followed by
         the current input and output block counts will be writ‐
         ten to the standard error output in the same format as
         the standard completion message.  If the sync ...

This seems to be exactly what I want/need on disk cloning, i.e., on error don't stop, but continue and drop an error message. If there would be an error, then I want dd to rescue as much of the data as possible. After cheking the error log, I could put that cloned disk safely aside, and try again to get a whole clone using recoverdisk(1)() onto a new disk.

Best regards

Rolf
 
rolfheinrich said:
dump(8)/restore(8) doesn't even remotely come close to a transfer rate of 70 MByte/s. A recent experiment with the same disks gave 15 MByte/s, and at the same time the processor utilization was quite high.

# mount -o async /dev/ada1p3 /mnt; cd /mnt
# dump -C 24 -0af - /dev/ada0p3 | restore -rf -
Click to expand...

That will go about two to four times faster if you give dump a -b64 flag.

The relevant section of dd(1)() tells:

Code: 
noerror  Do not stop processing on an input error.  When an
         input error occurs, a diagnostic message followed by
         the current input and output block counts will be writ‐
         ten to the standard error output in the same format as
         the standard completion message.  If the sync ...

This seems to be exactly what I want/need on disk cloning, i.e., on error don't stop, but continue and drop an error message. If there would be an error, then I want dd to rescue as much of the data as possible. After cheking the error log, I could put that cloned disk safely aside, and try again to get a whole clone using recoverdisk(1)() onto a new disk.
Click to expand...

If there's an obvious error... well, I still have mixed feelings about this.
 
Wow, lots of great info, thanks everyone.

So if I'm understanding this right, if I want to go the dd route I would do the following

Code: 
dd if=/dev/targetdrive of=/dev/backupdrive bs=1m conv=noerror

Then to restore the data from backup drive I would just reverse the process?

Code: 
dd if=/dev/backupdrive of=/dev/targetdrive bs=1m conv=noerror
 
For that to work, either the drives must be identical in size, or the partitions defined so they fit on the smaller drive. It must be MBR, not GPT. The better way to "restore" would be to simply disconnect the failed drive and connect the backup drive instead.

It's not really a backup, it's more of a copy from a while back. What I mean by that is there is only one copy. If you want to restore a file that was deleted before the most recent copy was made, it doesn't help. If you create a new directory and new dumpfiles each time, that file can still be accessed in the older files.

If the concern is the system disk dying, a mirror can be a better option. If the worry is about incremental convenience backup (recovering accidentally deleted files and old versions), see sysutils/rsnapshot. None of these is a replacement for an off-site backup, but they do go well with it.
 
wblock@ said:
... It must be MBR, not GPT. ...
Click to expand...

Are you sure?

I used dd for cloning GPT disks. The partitions are cloned exactly, and I can access the whole file structure on all the partitions. I have to admit that I did not try to restart from the clone. Is there something, that I need to be worried about?
 
If your source file systems are mounted writable, then both dump and dd have caveats, but I would personally trust dump a lot more than dd in that situation because dump has options to deal with backing up writable file systems. dd does not (and damage to the target from copying a changing file system with dd is unpredictable).

dd is fine if your disks are identical, and file systems not mounted (writable).

As others have indicated:
  • because dd does not care about file system data structures, it is provides certain emergency rescue options;
  • you may need to benchmark blocksize for both dump and dd to optimise speed; and
  • dump requires partitions and boot blocks to be restored manually.
dd has a place for special cases requiring disk duplication, but for regular backups I would always prefer dump.

Having said that, I have not used dump for many years. These days my backups go to disk (both local and remote) using rsync. It's *much* faster because once the initial copy is in place, only the changes need be backed up. Like dump/restore, rsync requires that you manually re-create partitions (and possibly boot block) when restoring.

Cheers,

Phil
 
rolfheinrich said:
Are you sure?

I used dd for cloning GPT disks. The partitions are cloned exactly, and I can access the whole file structure on all the partitions. I have to admit that I did not try to restart from the clone. Is there something, that I need to be worried about?
Click to expand...

Backup GPT partition tables have to be at the end of the disk, not before. If the disks aren't identical in size, one will have an incorrect backup GPT.
 
wblock@ said:
Backup GPT partition tables have to be at the end of the disk, not the before. If the disks aren't identical in size, one will have an incorrect backup GPT.
Click to expand...

Ah, OK!

I always use exactly the same hdd model for cloning. Next time, I will try dump/restore with -b64 as dump option.

Many thanks for the clarification and best regards

Rolf
 
gpw928 said:
Having said that, I have not used dump for many years. These days my backups go to disk (both local and remote) using rsync. It's *much* faster because once the initial copy is in place, only the changes need be backed up.
Click to expand...

rsync(1) can use hard links for things that haven't changed. rsnapshot(1) uses that and rotates copies, so it doesn't take much space.
 
What I'm trying to do is make a clone of my system after I've got everything setup I like (xfce, preferences, other ports, etc) basically "my base system" that way whenever I want to bring my computer back to square one I don't need to spend time installing FreeBSD, ports, changing files, etc.
 
So... a snapshot in time. But wouldn't it be nice to have more than one? Make a new directory like 2012-08-21 each time you do a backup, and you'll be able to go back in time.
 
I have two hard drives (of different sizes) installed in my computer. I clone my primary to my backup drive once in a while. I use this script to do it

Code: 
#!/usr/local/bin/bash

time dump -1 -a -L -f - / | ( cd /mnt/root ; restore ruvf - )
time dump -1 -a -L -f - /var | ( cd /mnt/var ; restore ruvf - )
time dump -1 -a -L -f - /tmp | ( cd /mnt/tmp ; restore ruvf - )
time dump -1 -a -L -f - /usr | ( cd /mnt/usr ; restore ruvf - )

Might be a decent starting point for you. Works for me, though it isn't the fastest thing. I have moved as many non essential files to a partition that I do not clone.
 
You must log in or register to reply here.
Back
Top