Re: ThreePointOne: Contacts

[Rob Taylor <rob taylor codethink co uk>]

On 19/04/11 13:45, Patrick Ohly wrote:
> Hello!
> 
> Rob Taylor <rob.taylor <at> codethink.co.uk> writes:
>> On 19/04/11 11:27, Alexander Larsson wrote:
>>> On Tue, 2011-04-19 at 11:43 +0200, daniel g. siegel wrote:
> 
> Ross pointed me to this discussion. Let me jump into the discussion by
> replying to a more or less random post and quoting some other relevant
> statements. I'm not subscribed, so please CC me.
> 
> For some background on why sync is hard, here's an article that I
> wrote a while ago for LWN.net:
> http://syncevolution.org/development/pim-data-synchronization-why-it-so-hard
> 
> Note that this applies to both contacts and calendar, and probably
> more. If you design something new for GNOME, please don't focus
> exclusively on contacts.
> 
>>>> another very important point is synchronisation. together with salomon
>>>> sickert we thought about how to solve this problem. basically we came up
>>>> with the idea of a self-replicating backend, like couchdb.
> 
> That only gives you a closed solution between peers which share that
> self-replicating data. If you model it after couchdb (or just stick
> with couchdb), then a unique ID is assigned to each new contact once,
> when it gets created, and thus there's never any confusion whether a
> peer already has some data.
> 
>>>> if we then
>>>> could add support to the contact apps of other computer/devices like a
>>>> n900 or android, we would get synchronisation and conflict management
>>>> for free.
> 
> Nothing will be for free once you want to add interoperability :-/ All
> of the challenges mentioned in the article above will still apply (no
> unique ID in those peers, legacy formats, different data models).
> 
>>>> then there is also the idea of having a webservice for the gnome
>>>> contacts app, where you can access your contacts over the internet.
>>>>
>>>> we are very interested in your opinions about this!
>>>
>>> I don't know really. Synchronization is a tricky subject, with complex
>>> protocols and risk for merge problems. Its almost always a source of
>>> weird problems. I don't think we want to have synchronization as some
>>> core part of the design. 
>>>
>>> On the other hand, its important that there is some level of support for
>>> synchronizing contacts with e.g. phones. So, I guess we need to think
>>> about where it fits in.
> 
> If I had one wish, then it is this one: ensure that each item gets a
> unique, vendor created ID that never changes throughout the lifetime
> of the item.
> 
> Currently EDS supports that for calendar (part of iCalendar 2.0
> semantic), but not in the file contact backend (not required by vCard
> 3.0). Any UID that might have been set in a new vCard is overwritten
> by a local ID.
> 
>> Hmm, its really not that tricky - the eventually consistent method used
>> by couch works well in most situations - all replicated endpoints
>> resolve to the same state without intervention, and if you want to pick
>> up any pieces, you can.
> 
> For couchdb<->couchdb sync, yes, but that's only a subset of the
> problem - unless you can convince the rest of the world to use the
> same system. Start by convincing the Tracker/Ontology proponents, who
> also want everyone else to use their system for data replication ;-)
> 
> I'm pragmatic and don't expect that any system will ever take over
> enough to use it exclusively.
> 
> Regarding couchdb: how complete is its data model? When it first
> showed up, SyncEvolution had some problems with it because REV wasn't
> supported, if memory serves me right.

Since 1.0 it's been pretty sweet, with all the functionality working
well - including live updates.

>> On supporting existing sync protocols, for 'traditional' sync, its
>> important to have one central point of resolution, and that's ideally a
>> server.
> 
> Then the server becomes easily the weakest link in the chain. In the
> past, dumb phones had a very limited data model that could be
> completely supported by a (SyncML) server. Nowadays the address book
> in a client is very rich and may contain data that is not supported by
> many servers - that's definitely the case with, for example, MeeGo to
> Google sync.
> 
> If the client then trusts the server exclusively to resolve conflicts,
> data is lost on the server.

Hmm, the actual problem here is not the server - its the representation
format. I would argue that the correct approach to this is to always
represent all the data you receive from clients - and transform the
stuff you do understand in to a common representation.

> Another conceptual problem is that the server cannot communicate well
> with the user. A client might pop up rich dialogs and ask the user for
> help in cases that it cannot decide automatically (but designing such
> a dialog is an unsolved problem). SyncML servers (and probably other
> servers, too) don't have that option and instead fall back to
> heuristics which fail inevitably in some cases.

Think of couch as a bit like git - you never resolve on the server.
you effectively get a forking history, with a consistent algorithm to
choose a winning version of history. That mean on the client side, you
can look see if there was a fork of history and resolve at a point that
makes sense in the user's workflow.

>> A nice plan would be a server side component (on gnome.org of
>> course) that has an activesync and syncml endpoint which syncs into a
>> couchdb database for replication with the desktop. For activesync,
>> z-push [1] works well, and can be made pretty scalable with a few
>> changes. for SyncML, there's good open source components like funambol
>> [2] available.
> 
> I personally would rather recommend the Synthesis engine instead of
> Funambol.  Much more light-weight and miles ahead Funambol regarding
> SyncML support and data merging (for example, supports suspend/resume
> and much smarter conflict resolution). See the article above.

Sounds good to me also :)

> daniel g. siegel <dgsiegel <at> gnome.org> writes:
>> and right here i think we shouldn't base on bad formats (vcard) and
>> sucking protocols (syncml). using json is a much better option.
> 
> And what protocol do you suggest to use? JSON is not a protocol.

I think he's suggesting couchdb's protocol.

Thanks,
Rob

> Anyway, your proposal again boils down to "let's build our own, not
> interoperable system". Yes, that's doable and even fairly easy, but
> doesn't solve the synchronization problem once other systems are
> involved.
> 
> You also mentioned "importing vCard" into such a system. To me,
> synchronization implies two-way data exchange and off-line data
> modification. Remove two-way and you end up with the much simpler (and
> limited) data import. Remove off-line and you can avoid conflicts, but
> at the cost of having to be online all the time when making changes. I
> don't find that satisfactory - see my blog post that Ross already
> pointed to:
> http://syncevolution.org/blogs/pohly/2011/state-syncing-open-source
> 
> Bye, Patrick
> 
> 
> _______________________________________________
> desktop-devel-list mailing list
> desktop-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/desktop-devel-list


-- 

Rob Taylor, CTO, Codethink Ltd. - http://codethink.co.uk
Twitter: @robtaylor78 - LinkedIn: http://www.linkedin.com/in/robtaylor78
Office: +44 161 236 5575 - Cell: +44 7891 533856