Re: [Tracker] Proposed future for Tracker - Smaller modules

[Martyn Russell <martyn lanedo com>]

On 08/09/14 15:56, Philip Van Hoof wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 8/09/2014 12:34, Martyn Russell wrote:
>
> Hi Martyn,

Hi Philip,

> > Module = tracker-store - ontology (database schema) - libstemmer
> > (stemming library used by libtracker-fts) - libtracker-common (type
> > and various utilities) - libtracker-data (database access and
> > implementation) - libtracker-fts (SQLite module for tokenising) -
> > tracker-store (daemon providing DBus interface)
> >
> > Module = libtracker-sparql - libtracker-bus (DBus and read/write
> > access to the DB) - libtracker-direct (readonly access to the DB) -
> > libtracker-sparql-backend (internal implementation) -
> > libtracker-sparql (public facing library to the DB)
>
>
> My proposal would be to keep tracker-store, ontology and
> libtracker-sparql together (as one project).

The reason I didn't put libtracker-sparql with tracker-store / core 
things is that logically it's quite a different thing for applications 
wanting to JUST run SPARQL queries.

I suppose it doesn't make all that much difference, you still require 
the store and other bits for libtracker-sparql to operate anyway.

I guess you could go as far as to say that you might want to JUST use 
DBus and none of the libtracker-sparql support at all, another reason 
for it being separate.

> The reason for that is that the actual goal of libtracker-sparql was
> to provide what libsqlite is for a RDMBS and SQL, but for RDF and SPARQL.

Yea, but libsqlite3 and sqlite are separate packages (actually I am not 
sure if they're separate projects at all), but having tracker-store does 
not mean you have to use libtracker-sparql.

> Without tracker-store, libtracker-sparql can't work.

It's more the other way round I had in mind, without libtracker-sparql, 
tracker-store works.

> By splitting the ontologies into a separate project, managed by the
> Nepomuk organization or not, we could someday refactor
> libtracker-sparql to support multiple instances of installed ontologies.

I wouldn't split the ontologies out, the store is no good without the 
database and without the schema it's even more useless :)

Besides, all the ontology validation and handling is in the libraries 
the store depends on.

> Ideal in that scenario would be that tracker-store becomes an impl.
> detail of libtracker-sparql (it'll manage the instances of stores on a
> on-demand basis).
>
> The only real three reasons of existence of tracker-store are:
>
>         - GraphUpdated
>         - The fact that SQLite isn't MVCC and we need WAL journaling
>           and checkpointing done by a separate 'writer' process

Yea, single point of update.

>         - Providing a ontology

Not sure I follow you here, the reason for the store is not to provide 
an ontology - at least libtracker-data does a lot of that stuff - I 
would have to double check this.

The store offers a lot of buffering, queueing and general "management" 
of updates (and queries). Let's not forget the DBus interface.

> When your process links with libtracker-sparql and uses direct-access
> mode, it effectively has everything it needs to deal with meta.db.

Indeed.

> The eventual dependency tree would look like this:
>
> Your program depends on:
> |
> +- nepomuk-ontology
> |
> +-libtracker-sparql
>    `- Internally uses libexec/tracker-sparql-store
>
> This API:
>
> https://developer.gnome.org/libtracker-sparql/stable/TrackerSparqlConnection.html
>
> Would look like:
>
> #include <nepomuk/2006-2008/ontology.h>
> #include <tracker-sparql.h>
>
> static void something ()
> {
>    GError *error = NULL;
>    Ontology *ontology = nepomuk_ontology_2006_2008_new ("session id");
>    TrackerSparqlConnection *con =
>         tracker_sparql_connection_get_for_ontology (ontology)
>    TrackerSparqlCursor *cursor = tracker_sparql_connection_query (con,
>                 "select ?a { ?a nie:title 'something in nepomuk' }",
>                 NULL, &error);
>
>
>    while (tracker_sparql_cursor_next (cursor, NULL, NULL)) {
>         g_print ("%s\n", tracker_sparql_cursor_get_string());
>    }
>
>    g_object_unref (cursor);
>    g_object_unref (con);
>    g_object_unref (ontology);
> }

Interesting idea.

> This would internally deal with starting and stopping a
> libexec/tracker-sparql-store, and no global "tracker-store" would be
> needed anymore (the fact that multiple users share the same "session
> id", creates the central storage for users of that "session id").
>
> Storage could go to:
> ~/.tracker/sessions/$session-id/nepomuk-2006-2008/meta.db or something

This is quite an addition to what we have now.

I don't think it makes much difference because libtracker-sparql will 
always depend on a tracker-store (or whatever alias you use for grouping 
components that update the DB - libtracker-data, libtracker-fts, etc.).

Consider the application that only wants to query and not update the DB 
- they don't want to depend on all the crap needed for updating the DB, 
just the raw libtracker-sparql part.

--
Regards,
Martyn

Founder & Director @ Lanedo GmbH.
http://www.linkedin.com/in/martynrussell