Re: [Tracker] Revisiting indexer/daemon architecture

[Martyn Russell <martyn imendio com>]

Ivan Frade wrote:
> Hi!

Hi :)

Thanks for writing our thoughts up on this Carlos. Perhaps I should
provide some further background. Yesterday Carlos, Ivan, Urho, Jurg,
Philip, Mikael and I all discussed on a conference call some of the
pitfalls with the current solution and what we could do to improve
performance and response times for the user. This coupled with idea that
we want to put the crawler and indexer together is the basis for this
email and the discussion.

> > trackerd:
> >      - Database read and writes
> >      - File system monitoring
> >      - Importing metadata to the database using the TTL file format
> >      - handle remote/virtual data
>
> I would move "File system monitoring" also to the indexer, so all the
> functionality related with the File system is in the indexer.
>
> Tracker-indexer will be just one provider of information, that push into
> tracker data about the filesystem, exactly in the same way evolution is
> an information provider that push information about emails.

It should be mentioned that doing this means the indexer will never die
and we will have 2 processes that are acting like daemons. The only
benefit here is that we don't have to send information between the two
processes about monitor updates only to receive the metadata changes
later in a TTL file. The extra process means more memory use and more
processes floating around.

Personally, I think this might be the way to go, but only because of the
communication requirements and traffic that would be required to keep
monitoring in a separate place.

I should also add that FTS support is very much in the forefront of our
minds while considering this major change to the architecture of the
daemon and indexer. We want to do FTS at the same time if possible (if
not before). We do have a roadmap which we are working to and this is it
(note this is a rough guide):

0.6.9x = novstore.
0.7.x  = vstore.
(*)    = optional.

w10 - Plan novstore/vstore switch for upstream/Maemo TRUNK (scrum)
w11 - Release 0.6.91 upstream
w12 - Preliminary changes from vstore branch (notation changes, etc)
w12 - Integrate stream analyser (*)
w13 - GNOME switch to git (this is happening in March, so w9-w14)
w14 - nostore feature freeze
w14 - Create upstream branch for nostore Tracker
w14 - Create Maemo branch for nostore Tracker
w15 - Prepare final vstore patch for upstream TRUNK
w16 - Merge vstore to TRUNK
w17 - FTS support/architecture rework
w18 - FTS support/architecture rework
w19 - FTS support/architecture rework
w20 - FTS support/architecture rework
...

I am guessing we will need at least 2 months with everyone committed on
this to get the basics in place. Bare in mind there will be a period of
testing and instability which is likely to follow too.

-- 
Regards,
Martyn