1

I have the requirement to sync files (documents mostly, sound bites, high quality images) between multiple PCs, Macs and Linux clients and servers.

The number of files is large and delta syncs would be required. I would like a file changed on anywhere to be replicated. Additionally offline access is required. Version Control would be exceptionally useful and graceful recovery from duplicates etc.

In short I'm looking for dropbox but not hosted, SaaS or any other 3rd party involvement. Live mesh from Microsoft (or what it's called this week) would be ideal but for again 3rd party.

I figure this must be out there with a webdav backend or such and I just haven't found the cleanest solution.

I'm looking for the best way to implement the above. Open Source tools preferred but not explicitly required.

Thanks all.

Antitribu
  • 1,719
  • 3
  • 23
  • 37

4 Answers4

3

What you describe sounds like a very general solution, which might be impossible (e.g. you cannot have both offline access and completely avoid manual merging, e.g. for multimedia files).

In addition to the other solutions, you might want to look at Unison. It will sync multiple directory trees to keep them identical after changes (particularly offline changes). Unlike rsync, it will actually cope with changes in multiple copies. It will intelligently merge them if possible, or prompt if it detects conflicts.

I regularly use it to sync my desktop to mobile devices, and it works really well.

It does need to be run periodically however, it is not a continuous service.

sleske
  • 10,009
  • 4
  • 34
  • 44
2

the environment you are trying to create could fast become very messy. what if multiple copies of a binary file changed at the same time in different places? How are you going to merge those changes?

You probably want to implement a (distributed) network file system of some sort. Versioning could be done with an underlying file system snapshots.

I am not sure if WebDAV will give you easiness and I/O you are looking for, but it could be so. Other alternatives are NFS, OpenAFS and CIFS.

monomyth
  • 971
  • 1
  • 5
  • 9
  • Messy yes I know, but it can be done, just look at drop box/live mesh for evidence. NFS, CIFS don't give local copies as such. I wasn't aware that OpenAFS had builds for Windows and Mac but will definitely be looking down that path. The Distrib file system is the idea; i was just hoping someone had a private backend for dropbox. – Antitribu Dec 04 '09 at 17:59
  • now, said all of this, I think maybe what you really want is a digital asset management solution (http://en.wikipedia.org/wiki/Digital_asset_management). and I don't know any open source solution there, it doesn't mean they don't exist. – monomyth Dec 04 '09 at 18:03
1

Use the popular rsync tool. It is multiplaform, mature and you'll get a lot of support.

nrgyz
  • 560
  • 2
  • 9
  • Sorry should have stated it in the question but I'm also looking for something that requires minimal interaction from users and isn't so much of a "cron job thing that gets run every minute". Similar to dropbox where files are detected as changed. – Antitribu Dec 04 '09 at 17:56
1

Sounds like a job for Subversion to me, what w/ the version control requirement. Had you not said that, I'd have probably suggested either rsync or Unison, but neither of those is going to help you with version control.

You can get nice graphical clients for Windows and MacOS (see http://en.wikipedia.org/wiki/Comparison_of_Subversion_clients for a long list of clients), and access via WebDAV is available.

Evan Anderson
  • 141,881
  • 20
  • 196
  • 331