suggested directory sync software

J

jims

I could probably write my own, but it'd be quicker if there was something that already existed.

I have my my documents folder, which I had in Windows, long before I ever learned about FreeBSD. I keep all my work there, and as systems get upgraded, I move it between them and try to manually keep it up to day. As you can guess, this is ANNOYING.

I figured, having a nice FreeBSD server, I can keep a master copy there, and then have the directories on other machines mounted, and keep them synchronized. The problem is, one machine is a notebook and not always connected.

What I would need:
- Synchronize two or more folders.
-- If two files are not identical between the folders
--- Use the most recent, if it's 'last edit' timestamp is at least 1 day different than the other, otherwise prompt.
--- if one file is missing in a directory, but recorded to have been there before, delete it from the others (prompting is fine, and probably desireable initially)
--- if one file is missing in a directory, and not recorded to have been there before, distribute it between directories.


Does anything like this exist and work well? I've checked the ports tree a couple of times, but am not sure what to search for.
 
Just registered to share my know-how in synchronizing my personal Data-Folder across multiple OSX, FreeBSD and Windows machines.

First of all when synchronzing between multiple hosts rsync is imho not a good choice - at least in the aforementioned scenario of OP, which closely resembles my setup.
Why? Rsync is a tool to synchronize files/folders from one host to another or vice versa, being just a unilateral synchronization. This is where rsync is excellent.

Working on multiple copies/machines and rsync'ing files/folders to the FreeBSD-Server (aka master) can easily lead to losing modified/new files, when one does not strictly run the rsync-job before shutting down the machine (or after a working session) - or one can simply forget to run the sync-job.
Furthermore you normally want to use the -d flag with rsync when you have deleted files to delete those files on master as well.

But in this scenario we need a bilateral (or multilateral) synchronization and here comes unison (https://www.cis.upenn.edu/~bcpierce/unison/) into play.

I use the following setup:
The FreeBSD-Server being the master (all hosts synchronize against this machine)
All other machines being slave (Two OSX-hosts, two FreeBSD, one windows)

When running unison you are prompted when changes (new, deleted, modified files/folders) are detected: Do you want to propagate those changes - it doesn't matter if there are new files on master oder new files on a slave, you decide for those items each, you are in complete control.

Unison works via ssh and setting it up involves basically the same steps as using rsync via ssh.

On the windows side a bit more work is needed (applies to rsync and unison as well):
Imho best practice is to install Windows Subsystem for Linux (aka WSL, f.e. the Debian variant) as you won't need to fiddle with windows tools.
Just configure WSL to get ssh working (culprit: the generated ssh-keys are not stored inside ~/.ssh but reside in C:/Users/USERNAME/.ssh which _should be /mnt/c/Users/USERNAME/.ssh - I'm not quiet sure, but I remember vaguely that I had to copy public and private key manually).

Tell unison which folder(s) to synchronzie and which to ignore, by using a profile file on each machine and there you go :)

My unison setup works for about 3 years now and I'm happily synchronizing between all my machines :)

Edit: ooops, just recognized that I dig up a ten year old thread - sorry for that...
 
fuss said:
Just registered to share my know-how in synchronizing my personal Data-Folder across multiple OSX, FreeBSD and Windows machines.

First of all when synchronzing between multiple hosts rsync is imho not a good choice - at least in the aforementioned scenario of OP, which closely resembles my setup.
Why? Rsync is a tool to synchronize files/folders from one host to another or vice versa, being just a unilateral synchronization. This is where rsync is excellent.

Working on multiple copies/machines and rsync'ing files/folders to the FreeBSD-Server (aka master) can easily lead to losing modified/new files, when one does not strictly run the rsync-job before shutting down the machine (or after a working session) - or one can simply forget to run the sync-job.
Furthermore you normally want to use the -d flag with rsync when you have deleted files to delete those files on master as well.

But in this scenario we need a bilateral (or multilateral) synchronization and here comes unison (https://www.cis.upenn.edu/~bcpierce/unison/) into play.

I use the following setup:
The FreeBSD-Server being the master (all hosts synchronize against this machine)
All other machines being slave (Two OSX-hosts, two FreeBSD, one windows)

When running unison you are prompted when changes (new, deleted, modified files/folders) are detected: Do you want to propagate those changes - it doesn't matter if there are new files on master oder new files on a slave, you decide for those items each, you are in complete control.

Unison works via ssh and setting it up involves basically the same steps as using rsync via ssh.

On the windows side a bit more work is needed (applies to rsync and unison as well):
Imho best practice is to install Windows Subsystem for Linux (aka WSL, f.e. the Debian variant) as you won't need to fiddle with windows tools.
Just configure WSL to get ssh working (culprit: the generated ssh-keys are not stored inside ~/.ssh but reside in C:/Users/USERNAME/.ssh which _should be /mnt/c/Users/USERNAME/.ssh - I'm not quiet sure, but I remember vaguely that I had to copy public and private key manually).

Tell unison which folder(s) to synchronzie and which to ignore, by using a profile file on each machine and there you go :)

My unison setup works for about 3 years now and I'm happily synchronizing between all my machines :)

Edit: ooops, just recognized that I dig up a ten year old thread - sorry for that...
Click to expand...
dear fuss :
your unison app solution is very good for me . thanks.
 
I used to use a RAID1 pair, but have since gone to two separate drives and a sync utility.
The 2nd drive on the server is beyond reach of user accounts, so less likely to get trashed by malware.

I have the sync set as a one-way type:
Date on Source is copied to Destination only.
No reverse sync.

I have found that an accidental directory erasure on Source can go undetected for some time.
Even if deleted on Source, the last copy of it is still present on Destination.
This does accumulate duplicate copies when a Source directory is renamed.
I have a separate solution for duplicates.
 
You must log in or register to reply here.
Back
Top