Thoughts On File Sharing

Update: Celso suggested tsyncd (which has some interesting aspects) and iFolder 3.1, which I've installed and found to still have the same issues (i.e., it's server-based, with a "lite", non-secure server hanging off /opt/novell, and the GUI asks me to create an account on a server before I do anything whatsoever).

I've been looking for something to relieve the hassle of using Subversion to keep my document tree updated across machines, and started re-visiting the issue of having a set of files transparently replicated wherever I might be. I haven't found any magical solutions yet, but my meanderings have been of some interest and I thought I'd jot down some notes.

Although I keep my VoodooPad (Lite) stuff in my .Mac's iDisk and can draft stuff on any Mac, I need something better, more cross-platform (Apple hasn't updated their Windows iDisk software in ages), more secure and, preferably, less dependent on any service provider.

My full requirements are somewhat non-trivial, but they can be summarized as follows:

  • Easy to setup and manage
  • Transparent filesystem access (i.e., it all has to look and feel like part of the filesystem, and adding files has got to be as simple as dropping them in)
  • Secure authentication and transfer
  • Firewall traversal
  • Private folders for my own use, replicated across machines
  • Per-group folders, shared only among a few people
  • Fully multi-platform (but no need for Mac OS X resource forks and similar idiocies)
  • Completely peer-to-peer. I don't want a single server (live replication also preempts the need for backups)

I've actually looked at P2P applications to see if there were any suitable for this, but having spent a while blocking those myself, the concept of re-using something as dicey as Gnutella as the underpinnings for my personal replication service feels a little awkward (but it's doable, so I might re-visit the issue soon).

If we lived in an ideal world I'd use something like Groove, but Microsoft's acquisition of it (and lack of any real news in Ray's blog, despite the recent move to MSN Spaces) means there will most likely never be a Mac version - plus I'd like to have a replica of my stuff in at least one Linux box...

Interestingly enough, Microsoft also acquired FolderShare recently, which means there might be an actual demand for this sort of thing (but makes me wonder if they found Groove's replication features lacking in some respect).

So I started looking, and, as usual, went way down into basics. Besides looking through the P2P junk, I also looked at filesystem stuff, libraries and commercial applications. The list is long, but here are the usual suspects:

  • OpenAFS is an old favorite (anyone who was at INESC in the nineties will remember it fondly), but it requires too much baggage and OS-level tinkering.
  • Coda would be nice (due to its disconnected operation features), but isn't readily available for Mac OS X.
  • iFolder is, at this point, largely useless, both because it's targeted at LAN environments and because it lacks a decent server.
  • pfig dropped me a link to MogileFS on my lap - it's interesting and (nearly) portable (it's Perl, which means it has a far better chance than most other stuff), but it's geared towards applications, not end-users.
  • Freenet is a bit too politically-minded (and wantonly distributed) for my taste, and far too raw to be of any real use (although it surely makes the cut where it regards security).

Oddly enough, I decided to check out JXTA, which has always been somewhat discreet (one would nearly call it "playing dead") in the CSCW field, and whose site suffers from an identity crisis. It's both an initiative and a showroom for stuff, but you have to dig in really deep to find anything even remotely useful.

There seem to be quite a few applications, but digging deeper revealed only a lot of statements of intent and a few concept demos, so it's actually pretty surprising that I was able to find WiredReach.

It's Java/Eclipse-based (and accordingly fast as molasses on my iBook) and I've yet to figure out exactly how open (and free) it is, but despite feeling a bit like Groove - i.e. it has the notion of a shared space - at this point it turns shared spaces into a sort of half-baked "shared blog", which is useless to me (and why the RSS reader, anyway?).

So I'm now wading through SourceForge to see if there's something out there that fits the bill, all the while wondering if there is any way I can get monotone to replicate stuff automatically.

Other suggestions are welcome - even if only to stop me from trying to wrap monotone in a bunch of Python scripts to try to do what I want.