FreeBSD Dump to usb drive

Place to ask questions about partitioning, labelling, filesystems, encryption or anything else related to storage area.

FreeBSD Dump to usb drive

Postby Understudy » 03 Feb 2013, 00:51

Hi,

I have looked through several threads here and on google and may have overdone it and caused my brain to melt.

Here is what I have.

Code: Select all
$ uname -a
FreeBSD spider.brendhanhorne.com 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:15:25
UTC 2012     root@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386



I have a live server. I want to do a complete dump to a usb hard drive.
The setup looks like this:

Code: Select all
$ df -h
Filesystem          Size    Used   Avail Capacity  Mounted on
/dev/mirror/root    358G     56G    273G    17%    /
devfs               1.0k    1.0k      0B   100%    /dev
/dev/da0s1          458G    162G    259G    38%    /usb



Now this is till early on so let me not freak anyone out. The /usb (da0s1) will be zero out before I get started.

What I want to do is a complete dump to the usb. One of my concerns is how to not cause a redundant loop. If I do a complete backup I don't want it to try and dump /usb to /usb. That would be bad as I understand it.

In addition I would like to do a incremental dump that will add only new or updated file information. I would add that as a cron job to a script.

So if I do the following am I screwed:

Code: Select all
# dump -OaLuf /usb /dev/mirror/root



I assume the follow up command is going to be something like:
Code: Select all
# dump -1aLuf /usb /dev/mirror/root


Sincerely,

Brendhan
My dream is to have a post that doesn't need to be edited by DutchDaemon.
Understudy
Junior Member
 
Posts: 83
Joined: 17 Nov 2008, 01:37
Location: West Palm Beach, FL

Postby jb_fvwm2 » 03 Feb 2013, 01:26

Early in my usage of FreeBSD, I repeatedly wrecked UFS filesystems writing to them on usb disk(s) (IIRC). (usb drivers, disk firmware, ... ?). I can write full-speed (not too much data at once) to a thumbdrive, read full-speed FROM the thumbdrive, gcp (with a [FILE]sleep 1[/FILE] programmed between a many-file write, but for backup purposes, I'd recommend the [FILE]rsync[/FILE] command with the [FILE]bwlimit[/FILE] parameter (1000 or 2000 or so, see the forum threads which contain [FILE]bwlimit.[/FILE]; it seems to go hayware much less often (sata devices and firmware...), and when it does, it is simple to [FILE]fsck_ffs -y [/FILE] (despite the SUJ journal disappearing, never to return sometimes, rendering the /dev/ad10.journal simple /dev/ad10, to [FILE]fsck_ffs [/FILE] the latter rather than the former... OTOH, I have not used USB drives in years, others could maybe answer with more direct solutions to the question.
Further, the usb drivers were IIRC entirely rewritten since that time (referenced in my first sentences above).
[color="DarkOrange"]Using [FILE]/lookat/[/FILE] with zsh/grep/find/aliases/pipes/portmaster and [FILE]/var/db/pkg/[/FILE] flat files to meteorically speed [/color] port installs/upgrades forever hopefully...
jb_fvwm2
Senior Member
 
Posts: 1503
Joined: 16 Nov 2008, 19:19

Postby wblock@ » 03 Feb 2013, 02:54

Understudy wrote:I have a live server. I want to do a complete dump to a usb hard drive.
The setup looks like this:

Code: Select all
$ df -h
Filesystem          Size    Used   Avail Capacity  Mounted on
/dev/mirror/root    358G     56G    273G    17%    /
devfs               1.0k    1.0k      0B   100%    /dev
/dev/da0s1          458G    162G    259G    38%    /usb



Now this is till early on so let me not freak anyone out. The /usb (da0s1) will be zero out before I get started.


That is not necessary. What filesystem is on the USB drive? Please don't say NTFS.

What I want to do is a complete dump to the usb. One of my concerns is how to not cause a redundant loop. If I do a complete backup I don't want it to try and dump /usb to /usb. That would be bad as I understand it.


It won't happen. [man=8]dump[/man] does not cross filesystems.

In addition I would like to do a incremental dump that will add only new or updated file information. I would add that as a cron job to a script.


[man=8]dump[/man] can do that, but it makes the backup more fragile. Restores take the original level 0 dumpfile plus all the ones made since then. I personally don't use anything but level 0.

So if I do the following am I screwed:

Code: Select all
# dump -OaLuf /usb /dev/mirror/root


No, but that won't quite work. [file]/usb[/file] is a drive. Write to a file on that drive:
Code: Select all
# dump -C16 -b64 -0aLuf /usb/backup-2013-02-02.dump /dev/mirror/root


[file]-C16 -b64[/file] will increase the speed.
User avatar
wblock@
Old Fart
 
Posts: 10222
Joined: 07 Sep 2009, 23:23
Location: Milky Way galaxy

Postby Understudy » 03 Feb 2013, 04:55

wblock@ wrote:That is not necessary. What filesystem is on the USB drive? Please don't say NTFS.


No it is ufs. I used it before to backup a freebsd 8 laptop that now has been upgraded. That is why I said this.

Now this is till early on so let me not freak anyone out. The /usb (da0s1) will be zero out before I get started.


It won't happen. [man=8]dump[/man] does not cross filesystems.



[man=8]dump[/man] can do that, but it makes the backup more fragile. Restores take the original level 0 dumpfile plus all the ones made since then. I personally don't use anything but level 0.



No, but that won't quite work. [file]/usb[/file] is a drive. Write to a file on that drive:
Code: Select all
# dump -C16 -b64 -0aLuf /usb/backup-2013-02-02.dump /dev/mirror/root


[file]-C16 -b64[/file] will increase the speed.


Okay I am liking this.

Here is the fun thing.

Code: Select all
spider# ls -a /
.               backup          home            proc            usb
..              bin             lib             rescue          usr
.cshrc          boot            libexec         root            var
.profile        dev             media           sbin
.snap           entropy         mnt             sys
COPYRIGHT       etc             nfs             tmp


Within the / is usb which if I execute the command makes me worry that it will go into a loop. Is there any cause for worry on that? I know you said it does not cross filesystems but the the usb is ufs.

My reason for doing an incremental [man=8]dump[/man] was that the usb drive is basically the same size as the hard drive on the server. So I figured as the server had more room taken up it did not have to do a complete overwrite each time. It would only do a dump of the new or changed items. From what I understand in your post you are saying it is easier to just do a daily dumps of a few hundred gigs of data?

Sincerely,

Brendhan
My dream is to have a post that doesn't need to be edited by DutchDaemon.
Understudy
Junior Member
 
Posts: 83
Joined: 17 Nov 2008, 01:37
Location: West Palm Beach, FL

Postby wblock@ » 03 Feb 2013, 07:16

The USB drive is mounted on [file]/usb[/file], but it is not part of that filesystem, it is a separate filesystem. A separate [man=8]newfs[/man] command was used to format it. So no, [man=8]dump[/man] will not cross from the filesystem being dumped over into other filesystems that are mounted.

As far as backup schemes, it really depends on what you are doing, what is available, and what the budget allows. If higher-level dumps are required, do that. It would be a good idea to practice restoring some sample files from a set of those backups. That's a good idea for any type of backup, for that matter.
User avatar
wblock@
Old Fart
 
Posts: 10222
Joined: 07 Sep 2009, 23:23
Location: Milky Way galaxy

Postby kpa » 03 Feb 2013, 14:12

I know you said it does not cross filesystems but the the usb is ufs.


The word "filesystem" has a few different meanings depending on context. A filesystem can be of certain type, UFS, FAT, ZFS etc. Or the word can mean data stored on the disk using a certain filesystem as files and directories and that filesystem mounted at a mountpoint. Wblock@ used the latter meaning here. In other words, [man=8]dump[/man] does not cross mountpoints.
kpa
MFC'd
 
Posts: 3402
Joined: 05 Jul 2010, 13:19
Location: People's Technocratic Republic of Finland

Postby Understudy » 03 Feb 2013, 14:42

My budget is I have a [man=8]gpart[/man] RAID 1 mirror on 2 drives. I have a usb drive.

I agree that doing restores is a good idea. I want to understand my dump methods properly first.

Baby steps.

Sincerely,

Brendhan
My dream is to have a post that doesn't need to be edited by DutchDaemon.
Understudy
Junior Member
 
Posts: 83
Joined: 17 Nov 2008, 01:37
Location: West Palm Beach, FL

Postby Understudy » 03 Feb 2013, 14:56

As I learn more I thought that [man=8]dump[/man] at 0 would make a full backup and than the incremental backup would update that dump file. The more I read on this the more it seems like that is not the case. Still learning.

Perhaps the idea under my current thinking is that I would have a full dump on Sunday and do incremental backups the other 6 days and than start the process over again. I would do this with a script and a [man=8]cron[/man]. Currently this is where my head is. Subject to more confusion as I continue to learn and ask questions.

Sincerely,

Brendhan
My dream is to have a post that doesn't need to be edited by DutchDaemon.
Understudy
Junior Member
 
Posts: 83
Joined: 17 Nov 2008, 01:37
Location: West Palm Beach, FL

Postby wblock@ » 03 Feb 2013, 17:10

[man=8]dump[/man] writes a new file each time. Level zero saves all files, higher levels only save files that have changed. So if you accidentally delete a file on Thursday, you have to go back through the dumpfiles looking for it. If it changed on Wednesday, it will be in that incremental backup. Otherwise, you may have to try each one all the way back to Sunday, the level zero dump.

[port]sysutils/rsnapshot[/port] works well as an addition to other backup. It uses [port]net/rsync[/port] and hard links to keep snapshots of directories you pick. Because of the links, it is highly space-efficient, only using more space for files that have changed. It can store the backup directories on other systems with SSH. Not a replacement for other backups, but a very nice accidental file deletion protection.
User avatar
wblock@
Old Fart
 
Posts: 10222
Joined: 07 Sep 2009, 23:23
Location: Milky Way galaxy


Return to Storage

Who is online

Users browsing this forum: No registered users and 1 guest