Re: New module proposal: tracker

[John Carr <john carr unrouted co uk>]

On Sun, Nov 8, 2009 at 8:43 PM, Iain <iain gnome org> wrote:
>> Tracker 0.7 ships with a DBus API, Nepomuk as ontology, SPARQL as query
>> language, SPARQL UPDATE for inserting and updating data and finally
>> signals-per-classes as life search capability.
>>
>> The DBus APIs are simple but very powerful thanks to SPARQL.
>
> From looking at tracker again this weekend, and indeed it does work
> better than the last time I checked in June (modulo some apparent big
> bugs and some issues with the concept that I couldn't test due to the
> bugs) but from a application developer point of view, I'm still
> disappointed.
>
> Essentially, I feel the APIs are far too low level and the
> documentation is very very incomplete, which ended up leaving me
> wandering around in dark as to what I was meant to do. The Application
> Development Manual is useless (I know its not finished, but its no
> help at all). I know you guys love love love ontolgoies and SPARQL and
> predicates and all that, but honestly, as an application developer I
> couldn't care less. All I want is to get at the data I want in the
> easiest possible way, and looking at tracker and discovering that I
> need to learn yet another language and a whole new load of boring
> concepts to do that is very offputting. SPARQL, tuples, ontologies
> should be internal details that you shouldn't need to know about.

While I agree that the terminology is a little buzzwordy and perhaps
even scary to newcomers, i want to start with a rather defensive
point: sqlite (or any database) isnt any better out of the box. I
think all C apps using sqlite are quite low level and SQL has to be
manually written? Things are better in python where we have ORM but
people often go low level anyway beecause they want to be faaaast.

But I agree. We can do better here.

In addition to the things Philip has mentioned you might want to look
at tralchemy, a python helper for working with tracker. Right now this
is an experimentation area for what we can build on top of the tracker
store. A couple of things it does:

 * It dynamically "binds" all the ontologies (schemas) tracker knows
to python objects similar to traditional python ORM tools. The
bindings don't needed to be updated for new ontologies, it just works.
 * This means simple queries can be as easy as Contact.get(firstName="John")
 * To add a new records is c = Contact.create(); c.firstName = "John";
c.commit()
 * Help works on classes: from tralchemy.ns import nco; help(nco.Contact)
 * Event propagation - Contact.connect("added", callback). Though this
might be disabled in HEAD pending some refactoring.
 * For more complicated queries, there is an experimental JSON
representation of SPARQL.

tralchemy has 2 cousins - jalchemy (JavaScript) and sparql-glib
(vala). They were developed to the point where they were useful for an
application but currently tralchemy is more actively developed (read:
experimented on). All of them can be found on labs.codethink.co.uk.

> Looking at tracker/src I see 7 things begining with libtracker- and
> none of them look overly helpful. The most useful looks like
> libtracker-client, but again its too low level. What I would really
> like to see is some simpler high level calls that enable me to get all
> data of (for example) Music, allow me to specify what fields I want,
> and it should be a GObject that I can connect to signals on to find
> out when a result has been added/removed/changed.

AFAIK, most of the libtracker- things are internal.

I believe there has been some discussion about how a dynamic gobject
binding might look, but no volunteers to write it yet. I would love to
see some proposals for what the API might look like, everything i can
think of severely limits the queries that are possible or is
excessively verbose. I wrote an example of the excessively verbose
kind, sparql-glib..

> We should be looking
> to make our APIs as simple and easy to use as possible to enable
> developers to make the applications without distracting them with
> irrelevant implementation details. If this already exists, I apologise
> and blame the lack of documentation.
>
> iain

John