UFS Backup - Page 2

**jadawin@** · #26 January 26th, 2009, 13:57

Why don't using something like backuppc?

varda · #27 April 19th, 2009, 14:12

Except of these ways I'm frequently using usual tar for backup purpose

Code:

# to file
cd /usr
tar --one-file-system -cjvf /path/to/usr-backup.tbz .

Code:

# mirroring entire partition
cd /usr
mount /dev/ad2s1f /mnt
tar --one-file-system -cf - . | tar -xvf - -C /mnt

Sometimes I'm using pax to clone disk after mounting new partitions respectively

Code:

cd /; pax -p e -X -rw . /mnt; \
cd /var; pax -p e -X -rw . /mnt/var; \
cd /usr; pax -p e -X -rw . /mnt/usr; \
cd /tmp; pax -p e -X -rw . /mnt/tmp

Sometimes I'm using cpdup utility for partitions synchronisation

Code:

mount /dev/ad2s1f /mnt
cpdup -vvv -x -i0 /usr /mnt

Those ways seems faster for me. I may be wrong.

If I want to backup/mirror over internet I'm using rsync or fuse sshfs. In local network I'm mounting storage from remote machine using ggated/ggatec and then everything as usual.

varda · #28 April 19th, 2009, 14:26

Forget to say about disk partitioning. I'm keeping it easy and simple.

Code:

# clear disk
dd if=/dev/zero of=/dev/ad2 bs=1m count=1
# initialize disk and create single slice
fdisk -BI /dev/ad2
# initialize slice and create single partition
disklabel -Bw /dev/ad2s1
# relabel slice to required partitioning
disklabel -R /dev/ad2s1 file
newfs /dev/ad2s1a
...

Manually creating simple partitons description file and using it in above snippet of code

Code:

#	size	offset	fstype	[fsize bsize bps/cpg]
a:	1G	16	4.2BSD		# /
b:	2G 	*	swap		# swap
c:	*	*	unused
d:	4G	*	4.2BSD		# /tmp
e:	8G	*	4.2BSD		# /var
f:	*	*	4.2BSD		# /usr

Sometimes enabling journaling

Code:

geom journal load
geom journal label /dev/da2s1f
newfs -J /dev/da2s1f.journal
mount -o async /dev/da2s1f.journal /mnt
# perform backup

May be adding it to fstab

Code:

# Device		Mountpoint	FStype	Options		Dump	Pass#
/dev/da2s1f.journal	/mnt		ufs	rw,async	2	2

This's just examples.

monkeyboy · #29 April 22nd, 2009, 21:17

the bigger question for me is just *what* to make backups onto... tape technology doesn't seem to have kept pace with disk tech.... but OTOH us ol'timers never did consider making "backups" onto disk to be "true" backups (must be archiveable, kept "forever", offsite, etc, etc.)

monkeyboy · #30 April 22nd, 2009, 21:22

Quote:

Originally Posted by monkeyboy

the bigger question for me is just *what* to make backups onto...

To pursue this a bit further... to make a "real backup" of a 1TB disk, which are available for about $100 these days, one needs a media form that can storage that 1TB of data costing no more than, say $10-20, such that you can stow away a copy "forever" every week or few weeks (at least 10-20 times a year). Is there such a storage technology?

graudeejs · #31 April 22nd, 2009, 22:33

I would rather make mirror (raid-some number i don't remember)
that way if disk goes down, than another will work.

And don't for get to buy good UPS

monkeyboy · #32 April 23rd, 2009, 01:36

mirror, RAID1, is good... for what it is... but completely different than a backup or snapshot/archive. rm * anyone?

furthermore if some software error corrupts or totally smashes that filesystem, guess what? you have TWO perfectly smashed copies of that filesystem...

graudeejs · #33 April 23rd, 2009, 06:21

Quote:

Originally Posted by monkeyboy

mirror, RAID1, is good... for what it is... but completely different than a backup or snapshot/archive. rm * anyone?

furthermore if some software error corrupts or totally smashes that filesystem, guess what? you have TWO perfectly smashed copies of that filesystem...

well then you're back to use of simple backup/restore, but probably with backup levels.
You don't want to save shitload of same data every time... and then once in a while do level 0 dump

fronclynne · #34 April 24th, 2009, 11:33

Quote:

Originally Posted by monkeyboy

To pursue this a bit further... to make a "real backup" of a 1TB disk, which are available for about $100 these days, one needs a media form that can storage that 1TB of data costing no more than, say $10-20, such that you can stow away a copy "forever" every week or few weeks (at least 10-20 times a year). Is there such a storage technology?

Heh. Good one. Last time I even looked at a tape, 80G was close to $90, and even assuming larger & cheaper, a weekly backup of 1T would take about two weeks.

Looks like a 15pack of 25G blah-rays is >$60. Man, it just gets worse all the time.

Have you looked at 3.5" floppies? They should be pretty cheap.

monkeyboy · #35 April 24th, 2009, 19:56

Quote:

Originally Posted by killasmurf86

well then you're back to use of simple backup/restore, but probably with backup levels.
You don't want to save shitload of same data every time... and then once in a while do level 0 dump

That's right... or at least one approach...

Quote:

Originally Posted by fronclynne

Heh. Good one. Last time I even looked at a tape, 80G was close to $90, and even assuming larger & cheaper, a weekly backup of 1T would take about two weeks.

Looks like a 15pack of 25G blah-rays is >$60. Man, it just gets worse all the time.

Well I'm currently using DLT IV 40/80GB tapes, which you can get on ebay for about $3/4 each. That's still 10X more expensive than I'd want (on a per TB basis), but worse, the quanta is too small. I'd settle for some kind of 200GB media at $100/TB (200GB real for $20).

I think this is a huge problem that, as you say, keeps getting worse. Namely that disk technology is far outstriping the development of suitable backup media. People buy these ultra-cheap 1TB disks thinking that they have 1TB of storage for cheap... but as any sysadm knows, your storage is only as good as your backup strategy.

Looking at the "big boys", (e.g. Network App., HP, "enterprise solutions"), they seem to be converging on "virtual tape libraries", which spread the architecture between disk and tape for backup. Mucho $$$ -- I haven't seen a OSS-type solution in the same vein (our university/health center spends millions $$$ on such things just to store perhaps a PB -- in other words they spend perhaps $10K to store 1TB, NOT $100).

Then there's the ol' "can you expect to read it in 10-20 years"... I have ALOT of 20 year-old data that is still very valuable, but the only offline data that I have that I expect to be able to read is on CDROM. I have other forms of archival media (8mm tape, MO), but the likelihood that I can read those things is pretty low, and 5-10 years from now, prolly close to zero.

I'd like to see some well-thought-out, AFFORDABLE, solution from the OSS community that addresses these issues. I think this is one of the hidden HUGE disservices that Microsoft has brought on to the computing world. MS has never considered proper backup/recovery to be an important part of computing. It is shameful that Windows never had a credible solution -- except perhaps those offered by 3rd parties. How many countless Windows users have faced a total data loss with "reinstall the OS" as their "recovery" solution -- pitiful.

graudeejs · #36 April 24th, 2009, 20:03

So what is so good about tape, that you still use it?
[I have never ever used/seen tape, I'm just a desktop user]

I don't seep problem keeping backups on HDD's, as long as you properly maintain them.

Btw what do you store on 1TB?

[lol, i know it's not much once you have it. My old PC had 8GB and i thought it was a lot. Now i have ~400GB, and i can't believe i could store my data on 8GB disk]

monkeyboy · #37 April 24th, 2009, 21:29

Quote:

Originally Posted by killasmurf86

So what is so good about tape, that you still use it?
[I have never ever used/seen tape, I'm just a desktop user]

I don't seep problem keeping backups on HDD's, as long as you properly maintain them.

Btw what do you store on 1TB?

Pls understand that my/our strategies are "in transition" and may not make total sense now vs when they may have started 5-10 years ago, when things were different -- hence these questions in this thread...

I store many GBs of medical research imaging data, as well as the usual documents, papers, presentations, etc.

I don't believe I have ever totally lost a single file in 30 years. And the times I have lost the most recent copy of a file I can count on one hand.

So tape... it USED to be that tape was 10-1000X cheaper than disk on a per byte basis. Its not that I particularly care for tape over disk backup. But I care about PROPER backups. Here is a casual list of some of the requirements for backup:

- ability to restore the state of a filesystem (or single file) across the whole lifetime of that filesystem (or file), sampled (snapshot) at certain intervals. I used to keep "monthly"'s FOREVER. I think keeping at least yearlies forever would be a minimum, with exponential-like frequency for more recent snapshots. For example, every year, plus every month for the past 2 years, plus every week for the past 2 months...

- ability to restore the system from scratch with only modest effort (not reading in 100 tapes/floppies/whatever).

- ability to read backups at least for 20 years.

- offsite/multisite storage, so that a fire/theft of a single site doesn't wipe you out.

then some niceties: minimal operator intervention, minimal time required, minimal down time of system, online backups possible, etc.

What's wrong with disk? maybe nothing... it depends on what you mean by "disk backup"... does it meet all the criterion listed above?

What's wrong with simply copying one disk to another?

- you only get one copy (or N copies, where N is very small), which means only N snapshots in time.
- there is appreciable chance that you will not be able to read current disks 10-15 years from now,
- not that easy to store at a separate site (stacks of hard drives on a shelf in another building?),
- and somewhat fragile to transport around...

My main points are:
- when one buys a 1TB disk for $100, that isn't the same as 1TB of PROPERLY backed up storage, which may cost more like $1000...
- people don't have a real appreciation for proper backups -- it used to be at any well-run data center, if you lost a file from a few days ago, they would be able to restore it for you without any question or issue...
- now that tape is no longer so cheap, relatively, there seems to be few, well-thought-out tools, strategies and software for affordable but solid backups...

Unix dump/restore was great... but what tools exist now to address the changing landscape in media (disk cheaper than tape), while retaining the same priorities and requirements of true backups?

graudeejs · #38 April 24th, 2009, 21:51

he he...
I only started backing up my system about half year ago....
before that, i never made a singe backup

probably it's because I still don't have to much valuable information.

I do backups mostly to save myself from recompiling FreeBSD over and over and over (not that i don't do that.... lol)

fronclynne · #39 April 25th, 2009, 08:51

Well, I just read to-day that the blu-ray gang are supposedly going to have 400G media on the market this year or next and 1T is supposed to be on the horizon. So, the ol' WORM jukeboxes reborn? In any case, so long as prices come down, it may be workable.

In the long term, I think archiving will be done in that area they're calling the "cloud" these days. I have medium confidence in what google is doing with fault-tolerance and redundancy. If it became possible to rent space from them (or amazon, or some other bunch of untrustworthy goons) reasonably and store data there no-questions-asked (your format, your encryption, their platters) it may well be as good as anything else that obviously won't matter when we're chained by the neck to some psycho on a motorbike.

graudeejs · #40 April 25th, 2009, 09:50

Blue-ray..... 1 scratch and .......

cloud computing....
Check "network security monitoring with freebsd" video clip in Freebsd community page

monkeyboy · #41 May 15th, 2009, 20:51

Quote:

Originally Posted by killasmurf86

So what is so good about tape, that you still use it?
[I have never ever used/seen tape, I'm just a desktop user]

I don't seep problem keeping backups on HDD's, as long as you properly maintain them.

Just in case anyone still thinks that making "backups" on a 2nd hard disk constitutes a real backup strategy... this article from Slashdot reminds us of why it isn't... and the subsequent discussion reiterates the points I was trying to make -- it ain't a "backup" unless it is 1) full, 2) on removable media, 3) offline, 4) offsite, 5) tested for actual restore...

These guys lost 13 YEARS of data cuz their "backups" weren't really backups...
===
Hacker Destroys Avsim.com, Along With Its Backups on Friday May 15, @01:17AM

"Flight Simulator community website Avsim has experienced a total data loss after both of their online servers were hacked. The site's founder, Tom Allensworth, explained why 13 years of community developed terrains, skins, and mods will not be restored from backups: 'Some have asked whether or not we had back ups. Yes, we dutifully backed up our servers every day. Unfortunately, we backed up the servers between our two servers. The hacker took out both servers, destroying our ability to use one or the other back up to remedy the situation.'"

graudeejs · #42 May 15th, 2009, 22:10

he he he, that's why i bought my 16BG flash

bluetick · #43 May 30th, 2009, 03:07

I notice the current man page for dump has added an example of dump to dvd.
/sbin/dump -0u -L -C16 -B4589840 -P 'growisofs -Z /dev/cd0=/dev/fd/0' /u

Mirror + rsync + dump = call me paranoid, but I do it.
Tape would be great but the price keeps it out of my reach.

Side note on dvd capacity article, 300 dvds on one disc.
http://news.bbc.co.uk/2/hi/science/nature/8060082.stm

graudeejs · #44 June 29th, 2009, 07:42

Quote:

Originally Posted by thortos

This strategy will probably fail for every server being used more than marginally. Especially dumping databases that are in use (such as Postgres or mySQL data directories) will yield inconsistent results and most likely result in non-working databases after recovery.

While I am aware that important databases are to be replicated live onto backup servers, I want to illustrate that this dump-while-in-use strategy is best used for desktops or low-profile servers, not for heavily-used systems.

How do you people handle the backups of your servers? I'm running a set of customized backup scripts per server that tar important directories and scp them to the backup server, starting and stopping daemons as needed, but obviously that's not for anyone with uptime requirements. I also have many servers running in VMware and use that to snapshot the VMs regularly and scp them to the backup server.

I think making ufs snapshot first and than dumping it, would solve the problem, because it takes very little time to make snapshot
http://forums.freebsd.org/showthread.php?t=3317

vsoto · #45 July 25th, 2009, 03:04

Quote:

Originally Posted by bluetick

/sbin/dump -0u -L -C16 -B4589840 -P 'growisofs -Z /dev/cd0=/dev/fd/0' /u

How can this command be modified to write a compressed backup to DVD?

V.

graudeejs · #46 July 25th, 2009, 11:26

Don't make simple things complex
Write a script if you need

Code:

$ dump -0Lauf - /dev/da0s1a | gzip >> /path/to/dumps/root.dump.gz
$ growisofs -Z /dev/cd0 -R -J /path/to/dumps/root.dump.gz

read
http://forums.freebsd.org/showthread.php?t=1195
about buring all kind of stuff to CD/DVDs
It explains how to write/append to cd/dvd and much more

jaymax · #47 August 24th, 2009, 01:10

An excellent HOW TO, but have I missed something - How to back up and restore the root directory, / . The /tmp, /var, /usr are straight forward once the / is established. Therein is my problem.

graudeejs · #48 August 24th, 2009, 01:59

boot from freebsd fixit cd (or freevsd dvd), newfs root, mount it and resotre it.
You can't restore root while it's used by base system.

jaymax · #49 August 24th, 2009, 06:32

Thanks!
It works
But one minor caveat on restoring another filesystem
You had a note

Code:

Important note: you need space in temp to be able to restore
if you run out of space in tmp, mount some filesystem somewhere and
create symbolic links from /tmp and /var/tmp to that mount point

what should be the space of the /tmp system relative to the dump file to be restored. I have a tmp filesystem 0.5 Gb that is linked to /tmp and to /var/tmp by linked directories /mnt/tmp/tmp and /mnt/tmp/tmp2 respectively. The file to be restored is ~ 240Gb the attempt produced a trail of "write failed filesystem is full" condition to stdout.

I changed the link(s) to another filesystem on another disk that had ~56Gb available space and ended up with the same error conditions.

BTW: I am using Fixit on the Installation CD

graudeejs · #50 August 24th, 2009, 08:48

what is the size of partition, that you are restoring?
I think it's smaller, then nessacery (I doubt it's because you don't have space on /tmp)

I haven't been dump/restore for some time now (using zfs now), but I remember (if my memory isn't failing), that when you run out of tmp, you'd get a lot of errors during restore (It took time for me to figure out, that I need more size on tmp)

Check if your partition is lorge enough.
Remember you're restoring ~240GB dump (I assume it's uncompressed size), that means that your FS must be larger.... (No I can't tell how large)

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

The Following User Says Thank You to monkeyboy For This Useful Post:
carlton_draught (April 14th, 2011)

The Following User Says Thank You to monkeyboy For This Useful Post:
graudeejs (May 15th, 2009)