Re: [Tracker] [Fwd: Plans for 0.7]

[Jamie McCracken <jamie mccrack googlemail com>]

On Tue, 2008-03-04 at 19:48 +0200, Ivan Frade wrote:
> Hi Jamie, 
>
>
> > 1) instead of tracker-index shared lib, only the parser would be shared
> > (because trackerd would need to parse and stem search terms). the search
> > api would not need to be shared nor would any other indexing stuff.
>
>  Do you mean to split the API in two components: search and index? I
> fixed it in the diagram. 

yeah thats pretty much it 

although there should be a link between libtracker-metadata and the
text index as we have to write to it when we set tags or any other user
defined stuff that needs to be indexed

>  I think it is just a question of how to compile the files.
>
> > 2) trackerd's access to db/index would primarily  be read only with
> > exception of ontology updates. for safe NFS use we must avoid separate
> > processes writing to same db at same time. so for storing metadata we
> > would need to  pause (via dbus) tracker-file-indexer if its indexing and
> > writing to metadata db (for performance reasons we could include option
> > not to do this and rely on file locking)
>
>  I suppose that we update the ontology only when trackerd is started.
> After that, we can launch tracker-file-index and both processes never
> write simultaneously in OntologyDB.


tracker-file-index will only ever access it read only so its not a
problem

> > 3) A User DB which is used for storing user defined metadata + services
> > (tags + any services stored directly in tracker as their primary store
> > like bookmarks and other stuff which is not stored elsewhere on disk)
>
>  This database is written by trackerd (when a client program uses the
> dbus-interface) and by tracker-file-index (e.g. to index a bookmarks
> file). So we could have some problems in NFS.
>
>  Here we can use your solution (stopping tracker-file-indexer) when a
> client program try to add contents in this database.

yes - even without NFS the transactions in the indexer process can block
access to sqlite so its best to freeze them out and force them to commit
their outstanding transactions if any (the current pause dbus method
does this already as does the search code which freezes the indexer
thread when you perform a search)

of course the file indexer will only ever be present when its indexing
files which wont be very often so there will be no need to do anything
when its not running and hence no overhead in most cases

jamie