Re: [Tracker] Database access abstraction

[Jamie McCracken <jamie mccrack googlemail com>]

On Tue, 2008-10-21 at 09:36 +0100, Martyn Russell wrote:

> > Also in the future i want to support direct access to sqlite via  a
> > client lib so we can bypass dbus (and trackerd) for select queries where
> > speed is paramount and volume of data is too big for dbus to handle
> > optimally (think get all my 100,000 music tracks with metadata). So this
> > library would have to handle all querying and any future ones (like
> > sparql) - so you will have no problem from me for implementing that
> > support in a lib
>
> Hmm, I would like to see the difference it makes using DBus and if it
> really is an issue. We have an API like this in DBus now which Phillip
> added - I really don't like the idea of people executing random SQL on
> the databases. It can lead to much bigger problems. Phillip stresses
> this in the .xml file where we document this API. I think quite rightly
> so too.  

that should be moved to a direct access lib

the advantage of a direcdt access lib is that it removes dbus overhead
when large amounts of data are required.

Rob taylor probably knows this better but from what i understand dbus is
not optimal for large payloads (~1mb+) and somehting like get all music
and metadata might involve a million plus strings which dbus would have
to marshall, strdup and validate individually into multiple packets
(IIRC packet size is 4kb?) so you are looking at massive overhead with
multiple ipc calls