Using ufs snapshots

graudeejs

Son of Beastie

Thanks: 690
Messages: 4,615

#1
Today I wanted to try out something new (file system snapshots)
It's pretty interesting method of making file backups, all tough backups are still required.


To demonstrate how snapshots work I will use virtual disk (This is 1st time I do it, and I don't want to experiment on my system, not to mention that I'm doing this over ssh)

I will show all steps as if I was root

-1) What is snapshot, what do I need to make one?

* You need UFS file system with enough free space.
If you run out of space during creation of snapshot, you will have problems (system may even start to panic)

* You need root access

* FreeBSD :D

0) Preparing Virtual disk
Code:
$ dd if=/dev/zero of=~/test.img bs=1 count=1 seek=10000000000
$ mdconfig -at vnode -u 1 -o noreadonly -f ~/test.img
$ newfs -U /dev/md1
1) Mounting the disk
to be able to make snapshots you need to mount disk with snapshot flag
to do it manually:
Code:
mkdir ~/mnt
mount -o snapshot /dev/md1 ~/mnt
to mount disk at boot time and be able to take snapshots add snapshot to flags field
example
Code:
/dev/ad0s1d		/usr	ufs		rw,snapshot 2 2
2) Copy some files to file system
Code:
$ cp -R /path/to/some/files1 ~/mnt/
$ cp -R /path/to/some/files2 ~/mnt/
$ cp -R /path/to/some/files3 ~/mnt/
$ cp -R /path/to/some/files4 ~/mnt/
3) Make snapshot
mksnap_ffs takes two arguments:
fs mount point and snapshot name

you can have up to 20 snapshots at any time per file system, and to remove snapshot all you need to do is simply find it and remove it with rm

Code:
$ mksnap_ffs ~/mnt fresh
Now you have your file system snapshot saved to fresh.

4) modify fs and make new snapshot
Code:
$ rm -R ~/mnt/files2
$ mksnap_ffs ~/mnt modifiedfs
now you have 2 snapshots of your filesystem.

5) What next?

when you have snapshots of file system you can fsck them, dump it, mount snapshot as if you fs was frozen in time :D

But the interesting thing is that you can recover your files after deleting them, or simply view them.

to restore fs from snapshot with dump/restore:
Code:
$ cd ~/mnt
$ rm -R files*
$ dump -0af - fresh | restore -rf -
that fs should only have snapshots

TIP: i suggest marking snapshots with nodump flag
Code:
$ chflags nodump ~/mnt/fresh
right after they are created
I made snapshot of 39GB fs in about 3 min (isn't that great?)

and using -h 0 switch when dumping snapshot
Code:
$ dump -h 0 -0af - fresh | restore -rf -
That way you can keep snapshots on fs when restoring fs from snapshot with dump/restore

A great thing about snapshot is that it can be burned to cd as regular file, and used later to restore fs.

It's very easy to make snapshots everyday with crontab.

finally when you don't need snapshot simply rm it.

Also when using snapshots you will (in most cases) need much less space compared to when using dumps


I made snapshot of 70GB fs (39GB used) in about 3 min (isn't that great?), and because fs wasn't modified, it's still only 39GB used :D

All in all, I'm happy I tried snapshots. Now I will need to redesign my FreeBSD setup a bit, to take full use of this wonderful technology. I believe it will make my life so much easier

One more thing. You can only save snapshot to same device you're making snapshot of.

Resources:
Handbook >> snapshots
mksnap_ffs(8)


P.S. I hope what i wrote makes sense to you. lol
If you need help, ask here, I'll try to help

EDIT:
you can't make snapshot of root fs
Don't add snapshot in /etc/fstab for root or you won't be able to boot in multiuser mode


EDIT 2:
You can use labels with snapshots :D

EDIT 3:
Instead of copying snapshots to some other media, I suggest extracting dumps and compressing them with gzip. That way you will save a lot of space, because when you make snapshot and copy it to other media it will be same size as original file system.
Dump will be significantly smaller in most cases. And it will be much faster to compress.
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 690
Messages: 4,615

#3
dinoex@ said:
I create snapshots via cron or shell,
and have the snapshots mounted as read-only fs.

This can be exported via samba so users can access the last backups
from their workstations.

Scripts:
http://people.freebsd.org/~dinoex/batch/snapshots/
if you copy snapshot to other fs, and $ chmod +x it, then you can mount it rw, this could be useful for jails, couldn't it?.

I haven't tried doing that on snapshot that is on original fs (i won't try for now)
 

tobe

Member

Thanks: 9
Messages: 92

#4
Hi,

I'd like to use snapshots to make remote backups via http.

Basically:

Every day, take a snapshot of the filesystem used to store all backups. Each snapshot will be removed after 7 days.

Implement a cgi script that will 'dump | compress | encrypt | http'.

The goal is to be able to download an encrypted backup at any time without having to create the encrypted file on the backup partition (it would take too much space). As the file can be pretty huge, it can take quite some time to download, that's why i want keep each snapshot for seven days.

Obviously, i don't want to run the cgi script as root or in the group operator, but the mksnap_ffs man page say:
The group ownership of the file is set to ``operator''; the owner of the file remains ``root''. The mode of the snapshot is set to be readable by the owner or members of the ``operator'' group.
It is safe to 'chmod 444' a snapshot ?

I mean, i did it and it works, but since a snapshot is not a normal file, can it cause some troubles ?

Thanks,
Tobe
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 690
Messages: 4,615

#5
I think it should be safe to $ chmod ugo+r
But i can't say it is, and take responsibility on myself for losing your data (in case it happens, somehow)

as i understand snapsots it would be DANGER to write to them (however haven't tested yet), until you copy them to other filesystem (i tried this)
 

tobe

Member

Thanks: 9
Messages: 92

#6
killasmurf86 said:
I think it should be safe to $ chmod ugo+r
But i can't say it is, and take responsibility on myself for losing your data (in case it happens, somehow)
Well i'm going to try on my own home server before crashing my company server :)
killasmurf86 said:
as i understand snapsots it would be DANGER to write to them (however haven't tested yet), until you copy them to other filesystem (i tried this)
Writing to a snapshot just doesn't make sense :)
Copying them to another filesystem just create an image of the source filesystem at the time the snapshot was created, so you can burn it on a DVD or on a tape.

I'm writing the cgi script in python right now, it's just going to pipe a /bin/sh script and transform the ouput to chunked http (the size of the body cannot be determined since it's packed on the fly).
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 690
Messages: 4,615

#7
tobe said:
Well i'm going to try on my own home server before crashing my company server :)

Writing to a snapshot just doesn't make sense :)
Copying them to another filesystem just create an image of the source filesystem at the time the snapshot was created, so you can burn it on a DVD or on a tape.

I'm writing the cgi script in python right now, it's just going to pipe a /bin/sh script and transform the ouput to chunked http (the size of the body cannot be determined since it's packed on the fly).
why not to run dump on snapshot, and send either dump, or gzipped dump over ssh?
more efficient
 

tobe

Member

Thanks: 9
Messages: 92

#8
killasmurf86 said:
why not to run dump on snapshot, and send either dump, or gzipped dump over ssh?
more efficient
Because i want to make it easy for anybody in my company to download the encrypted backups at home. They just need a web browser :)

Also, if i create a dump of the snapshot before sending it, it will take lots of space on the server (normally this is done directly on the tapes, not on the harddrives).
 
OP
OP
graudeejs

graudeejs

Son of Beastie

Thanks: 690
Messages: 4,615

#9
ok, but i hope you do realize that if you make snapshot for 50GB drive you will have to transfer 50GB data over net.
And i also hope your server has restore connection support... because if your boss is downloading and connection is lost at 49GB...... you get that point
 
Top