Re: [Evolution-hackers] Contacts database modifications history

[Patrick Ohly <patrick ohly gmx de>]

On Wed, 2014-04-16 at 12:23 +0200, Milan Crha wrote:
> On Tue, 2014-04-15 at 13:16 +0000, Potrola, MateuszX wrote: 
> > I would like to have ability to receive some kind of notifications (or
> > store information in some additional database) about modifications
> > made to contacts database during synchronization.
>       Hi,
> the compose of a "diff" for done changes is probably the hardest thing
> on this whole thing you'd like to achieve. There is always (at least)
> one place where this "diff" should be computed. I'm not much willing to
> add it into evolution-data-server itself, because it has no good use for
> it (at least not as of now).

I tend to agree. It would be very hard to come up with a general purpose
diff that works across a wider range of use cases. If it is
configurable, the config language and implementation would have to be
unnecessarily flexible (and thus complicated) for what Mateusz probably
has in mind.

I still think a plugin doing the diff would be okay.

> As Patrick pointed out, the previous implementation had sever issues
> which made the feature basically unusable. If we cannot make it done for
> all backends, then it probably might not be part of the
> evolution-data-server.

We have a slightly different understanding of EDS here. You see EDS as
an abstraction layer that offers just the functionality that can be
implemented with all backends.

We focus on EDS + local storage and try to make the best use of its
capabilities, even if that means adding things to it which won't work
with other backends.

That's find, I think we can do both. It just means that whatever API
gets added that only works with local storage needs to be clearly marked
as "local only" in the documentation and/or there has to be a discovery
mechanism for determining whether a  certain EBookClient instance
supports it.

> Anyway, I would prefer if you could create a plugin to SyncEvolution
> (does it support plugins?), also because it's the software you are
> targeting, not evolution-data-server code,

Not necessarily. At the moment the focus is on cached PBAP address books
that are only written by SyncEvolution, but that may change in the
future.

Perhaps someone wants to rip out SyncEvolution and do the caching
differently, or wants to have address books that can be edited by the
user.

The change tracking still needs to work in those cases.

Another argument for doing it inside EDS is that it can be combined with
the actual sqlite database update. To me, that is the only bullet-proof
way of ensuring that the change tracking information will be consistent
with the actual content. This must work in a car, where the head unit
can loose power at any moment. Ensuring that the sqlite database doesn't
get corrupted is difficult enough. Ensuring that different processes get
a chance to write different files will be even harder.

I'd like to mention further use case for allowing hooks during the
execution of EDS writes: this can be used to pre-compute data that is
later going to be shown by a UI other than Evolution. I think EDS
already does some of this for Evolution (like computing a full name) in
a way that, strictly speaking, depends on the UI that is going to show
the data.

One even crazier idea was to maintain an on-disk data structure that can
be memory-mapped to access contact data, including strings and their
parsed photos ready to be displayed, without going through sqlite and
all the string handling. I'm not a terribly big fan of that idea myself,
but if there were hooks in EDS which could be used to track DB changes
reliably, that would allow implementing this idea without further
changes to EDS itself.

> Still, all this looks too complicated, adding unnecessary burden,
> consider that you specifically target SyncEvolution changes, thus why
> not to extend SyncEvolution to write its own backlog of done changes in
> some place, where your application may just read them from?

SyncEvolution indeed does a read/update/write cycle when updating
contacts. This would be a natural place to add a hook for such a plugin.
Implementing it will be a bit tricky because it happens under the hood
in the Synthesis sync engine, but I guess it could be done, if the
alternative (a similar hook in EDS) is not acceptable.

> On Wed, 2014-04-16 at 09:02 +0000, Potrola, MateuszX wrote:
> > What is the probability of failure of emitting signals ?
>
> The probability is close to 0. If it happens, then it's a bug in the
> backend (or the related code) and should be fixed as soon as discovered.

The more likely failure is that EDS emits the change, but then the
computer is shut down before the receiving side gets a chance to handle
it => EDS DB was modified, change log in the receiver was not.

Bye, Patrick