File level retention

T

tcn

Hi,

I have been searching for a file level retention solution for my FreeBSD 10 server.

The system's data is on ZFS. I know I can use snapshots but from what I understand, it will take a whole disk or partition, not files and that I have to create them periodically. I would like an automatic snapshot of a file when is deleted so I can retrieve it later if required.

I have seen this in NAS systems (QNAP for instance has this feature). Is there a way of doing this?

Thanks,

tcn
 
I know that this isn't exactly what you are looking for but it's worth mentioning that you can technically access the .zfs/snapshots directory in a ZFS file system and dig around for a single file. It's not very robust but can save the day if you have to just dig up one file. It's not as elegant as what you are asking for though with a purely file based system. There's likely a backup solution out there that does this cleanly.
 
This is a lot of overhead... I thought something could be done since it is already on other systems. I didn't look in the FreeNAS project though, maybe something exists in there that would be brought into FreeBSD if not already there...
 
What overhead? It's a Perl program that uses net/rsync with hard links. Only runs on the directories you choose, only copies changed files, only takes space for changed files, much like ZFS snapshots.
 
Hmmm, I thought it would take the whole partition again; sorry, my mistake. I have too look into this more in depth then.
 
If there's anything in the FreeNAS project it will most likely be based on ZFS snapshots unless there is a 3rd party plugin for it.

Just to clarify the ZFS snapshots logic, they are not based on a 'disk' or a 'partition'. You take a snapshot of a ZFS dataset. If you have a directory containing all your data and that's all you care about, then you can make that directory into its own ZFS dataset and snapshot just that on its own.

Also when you create a snapshot, it doesn't 'take' anything. Nothing changes at all* when you take a snapshot and no data is copied anywhere. It just means that if you update or delete any data in the future, the old copy is kept around. Because ZFS is permanently copy-on-write anyway, there is no overhead at all other than the extra disk usage (which is rarely a problem unless you have constant heavy updates/deletes going on).

If you're already on ZFS I would personally just use one of the available ZFS snapshot manager scripts/ports and set it to keep a few hourly/daily/weekly/monthly snapshots. As mentioned if you need to restore a single file you can just copy it from ~zfs-mount-point/.zfs/snapshot/snapshot-name/path/to/file.

*Well apart from ZFS creating an internal reference for the snapshot obviously.
 
Right. ZFS snapshots only take up space as contents change. sysutils/rsnapshot is a lot like that, too. It hardlinks to existing unchanged files, so they only take up more space when the file changes. But it also keeps hourly, daily, weekly, and monthly copies. Very nice when there's a slip and someone modifies or deletes a file unintentionally.
 
Okay. @usdmatt is right; FreeNAS uses ZFS snapshots from what I can read. I have to really understand ZFS; I am using something I don't really know. All I know is that I can create RAIDs and I can easily add/remove drives to the pool. I was seeing snapshots as a complete dump but I see it isn't.

sysutils/rsnapshot seems like a nice tool too. Hard to tell which one fits my needs. From what I understand, ZFS snapshots are incremental and it is only when removing old snapshots that data is merged. But what if the files do not change? Diff is zero? Same behavior as sysutils/rsnapshot?
 
Last edited by a moderator:
ZFS snapshots are built around the Copy On Write architecture. So snapshots are really free, it just marks a spot in time. Files that are changed after that point take up more space.
 
You must log in or register to reply here.
Back
Top