Re: [Tracker] Database access abstraction

[Jürg Billeter <j bitron ch>]

On Mon, 2008-10-20 at 15:07 -0400, Jamie McCracken wrote:
> On Mon, 2008-10-20 at 20:07 +0200, JÃrg Billeter wrote:
> > as a preparation to support decomposed database tables in Tracker, I've
> > been looking into how we could abstract database access so that the
> > database schema can change without affecting the code in trackerd and
> > tracker-indexer.
>
> its also likely to affect the stored procs quite heavily and more so
> than the code.
>
> I dont think a full abstraction is necessary as saving/updating data is
> only done in a few places and this code should be shared more

I expect stored procedures to be less useful when moving to decomposed
tables as the SQL queries will depend on the schema/service definitions
and will therefore become more dynamic. Or do you have a plan how to use
stored procedures effectively with a database schema that depends on the
service definitions?

> > I'm proposing to introduce an additional (private) library that acts as
> > a high-level database interface, so it sits between libtracker-db and
> > trackerd/tracker-indexer. That library, let's call it libtracker-data
> > for now, is the only place where SQL queries get constructed.
>
>
> well I have been asking for a libtracker-metadata to host shared
> metadata support between trackerd and tracker-indexer so to some degree
> you will have my approval :)
>
> Also in the future i want to support direct access to sqlite via  a
> client lib so we can bypass dbus (and trackerd) for select queries where
> speed is paramount and volume of data is too big for dbus to handle
> optimally (think get all my 100,000 music tracks with metadata). So this
> library would have to handle all querying and any future ones (like
> sparql) - so you will have no problem from me for implementing that
> support in a lib
>
> Im not sure why you want to abstract *all* db access? i would have
> thought indexer specific requests can quite nicely remain where they are
> unless you have a good reason?

If only one library needs to know how the data is stored in the
database, it should be easier to do changes. I think this gets more
important with decomposed tables as we will have a less uniform schema
which will make building SQL statements more complicated, so it makes
sense to move it to a separate library, in my opinion.

> my preference is for sharing more routines rather than abstracting them

Abstracting is not really more than sharing with an appropriate
interface. My goal is not to write an over-engineered abstraction layer,
just enough abstraction to allow moving to decomposed tables. That's
also why I'm proposing to do this incrementally and refactor as needed.

JÃrg