Re: New module proposal: tracker

[Philip Van Hoof <spam pvanhoof be>]

On Tue, 2009-08-18 at 17:36 +0200, Maciej Piechotka wrote:

[CUT]

> Hmm. Is it one-in-all database? Then:
> - How you keep it out of corruption. Hardware and software errors
> happens and sometimes one lost files. If it is one file - ok I can live
> with it. If it is one-in-all file - ops (and please note that average
> user does not make backup).

We have something in mind for this where we'll have a automatic
periodically made backup of all of the user's data, plus a journal of
not yet backupped such data. This indeed means redundantly storing, but
the redundant store ensures that data-loss is virtually impossible (only
when both databases get corrupted simultaneously).

For that last rather unlikely event, we'll also have normal backup
features. The format of that backup will be Turtle, which is a humanly
readable w3 standardized format for storing RDF triples.

We have also plans to separate indexable data from user unique data, and
store the two kinds of datas in different locations (.cache and .local).

> - Some fs makes operations with small files much more efficient then
> bigger (reiser* for example). It may have performance impact.

We use sqlite which has its own internal performance optimizations that
work mostly unaffected by such FS specifics. The FS can of course always
have an influence.

Use a FS that makes sense, if it does (many apps use sqlite nowadays,
any FS that disses sqlite is going to perform badly for a lot of desktop
applications. Including Firefox and Evolution, for example).

> - What with concurrent access? 

Currently we give priority to read queries and we queue write queries.
Read queries are always solved immediately. We don't yet support having
two concurrently running read queries (we want to avoid usage threads in
tracker-store as much as possible, everything happens on the GMainLoop
at this moment). It's not decided that this wont ever change.


The Turtle format:
http://www.w3.org/TeamSubmission/turtle/


-- 
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://pvanhoof.be/blog
http://codeminded.be