Re: ThreePointOne: Contacts

[Patrick Ohly <patrick ohly gmx de>]

Hello!

Rob Taylor <rob.taylor <at> codethink.co.uk> writes:
> On 19/04/11 11:27, Alexander Larsson wrote:
> > On Tue, 2011-04-19 at 11:43 +0200, daniel g. siegel wrote:

Ross pointed me to this discussion. Let me jump into the discussion by
replying to a more or less random post and quoting some other relevant
statements. I'm not subscribed, so please CC me.

For some background on why sync is hard, here's an article that I
wrote a while ago for LWN.net:
http://syncevolution.org/development/pim-data-synchronization-why-it-so-hard

Note that this applies to both contacts and calendar, and probably
more. If you design something new for GNOME, please don't focus
exclusively on contacts.

> >> another very important point is synchronisation. together with salomon
> >> sickert we thought about how to solve this problem. basically we came up
> >> with the idea of a self-replicating backend, like couchdb.

That only gives you a closed solution between peers which share that
self-replicating data. If you model it after couchdb (or just stick
with couchdb), then a unique ID is assigned to each new contact once,
when it gets created, and thus there's never any confusion whether a
peer already has some data.

> >> if we then
> >> could add support to the contact apps of other computer/devices like a
> >> n900 or android, we would get synchronisation and conflict management
> >> for free.

Nothing will be for free once you want to add interoperability :-/ All
of the challenges mentioned in the article above will still apply (no
unique ID in those peers, legacy formats, different data models).

> >> then there is also the idea of having a webservice for the gnome
> >> contacts app, where you can access your contacts over the internet.
> >>
> >> we are very interested in your opinions about this!
> > 
> > I don't know really. Synchronization is a tricky subject, with complex
> > protocols and risk for merge problems. Its almost always a source of
> > weird problems. I don't think we want to have synchronization as some
> > core part of the design. 
> > 
> > On the other hand, its important that there is some level of support for
> > synchronizing contacts with e.g. phones. So, I guess we need to think
> > about where it fits in.

If I had one wish, then it is this one: ensure that each item gets a
unique, vendor created ID that never changes throughout the lifetime
of the item.

Currently EDS supports that for calendar (part of iCalendar 2.0
semantic), but not in the file contact backend (not required by vCard
3.0). Any UID that might have been set in a new vCard is overwritten
by a local ID.

> Hmm, its really not that tricky - the eventually consistent method used
> by couch works well in most situations - all replicated endpoints
> resolve to the same state without intervention, and if you want to pick
> up any pieces, you can.

For couchdb<->couchdb sync, yes, but that's only a subset of the
problem - unless you can convince the rest of the world to use the
same system. Start by convincing the Tracker/Ontology proponents, who
also want everyone else to use their system for data replication ;-)

I'm pragmatic and don't expect that any system will ever take over
enough to use it exclusively.

Regarding couchdb: how complete is its data model? When it first
showed up, SyncEvolution had some problems with it because REV wasn't
supported, if memory serves me right.

> On supporting existing sync protocols, for 'traditional' sync, its
> important to have one central point of resolution, and that's ideally a
> server.

Then the server becomes easily the weakest link in the chain. In the
past, dumb phones had a very limited data model that could be
completely supported by a (SyncML) server. Nowadays the address book
in a client is very rich and may contain data that is not supported by
many servers - that's definitely the case with, for example, MeeGo to
Google sync.

If the client then trusts the server exclusively to resolve conflicts,
data is lost on the server.

Another conceptual problem is that the server cannot communicate well
with the user. A client might pop up rich dialogs and ask the user for
help in cases that it cannot decide automatically (but designing such
a dialog is an unsolved problem). SyncML servers (and probably other
servers, too) don't have that option and instead fall back to
heuristics which fail inevitably in some cases.

> A nice plan would be a server side component (on gnome.org of
> course) that has an activesync and syncml endpoint which syncs into a
> couchdb database for replication with the desktop. For activesync,
> z-push [1] works well, and can be made pretty scalable with a few
> changes. for SyncML, there's good open source components like funambol
> [2] available.

I personally would rather recommend the Synthesis engine instead of
Funambol.  Much more light-weight and miles ahead Funambol regarding
SyncML support and data merging (for example, supports suspend/resume
and much smarter conflict resolution). See the article above.

daniel g. siegel <dgsiegel <at> gnome.org> writes:
> and right here i think we shouldn't base on bad formats (vcard) and
> sucking protocols (syncml). using json is a much better option.

And what protocol do you suggest to use? JSON is not a protocol.

Anyway, your proposal again boils down to "let's build our own, not
interoperable system". Yes, that's doable and even fairly easy, but
doesn't solve the synchronization problem once other systems are
involved.

You also mentioned "importing vCard" into such a system. To me,
synchronization implies two-way data exchange and off-line data
modification. Remove two-way and you end up with the much simpler (and
limited) data import. Remove off-line and you can avoid conflicts, but
at the cost of having to be online all the time when making changes. I
don't find that satisfactory - see my blog post that Ross already
pointed to:
http://syncevolution.org/blogs/pohly/2011/state-syncing-open-source

Bye, Patrick