Re: [Tracker] more issues with indexer-split

[Jamie McCracken <jamie mccrack googlemail com>]

On Tue, 2008-08-12 at 18:50 +0200, Carlos Garnacho wrote:
> Hi!,
>
> On lun, 2008-08-11 at 22:49 -0400, Jamie McCracken wrote:
> > As I will be working quite extensively on trunk post merge I require any
> > major changes to be done ASAP
> >
> > list of changes required prior to merge (in order of priority) - all of
> > these already exist and work in trunk:
> >
> >
> > 1) ïtrackerd: Handle file moves - update files in a directory
> > recursively when a directory is renamed/moved (need to pause indexer
> > before updating - watch out!). Likewise re-enable update of index from
> > trackerd as its needed for tagging and other user metadata
> >
> > 2) I could not see deletion of deleted and junk emails at startup (I see
> > it for new emails only but not for older emails that may have been
> > marked) - pls restore this functionality from trunk. Code needs to check
> > all summary files from start to end for junk and if not already marked
> > as junk in the JunkMail table then you must delete them and mark them.
> > If in doubt see trunk - older emails marked as deleted/junk will remain
> > in index which is unacceptable. see function check_summary_file in
> > trunks tracker-email-evolution.c
>
> This item is done in a different way:
>
> When trackerd notices the summary file has changed (or does the initial
> files processing), notifies the indexer. 
>
> The indexer, which is the one aware of the actual file contents,
> iterates through the messages, and the mail module will return NULL
> metadata for any junk/deleted email, making tracker-indexer delete
> anything in the DBs/index related to these mails.


that sounds inefficient - trunk only ever checked for existing deleted
or junk emails at startup because iterating through all emails in the
summary files is expensive. the use of a separate junk email table meant
lookups were confined to that table and not the services table so was
faster when number of emails was high

we should also avoid doing this whenever the summary file changes which
is why we stored an offset in trunk so we skip over messages to get to
the new ones only when summary files change or do nothing if no new ones
are present

the trunk way is faster so i would prefer that restored

thanks

jamie