Re: [Tracker] more issues with indexer-split

[Jamie McCracken <jamie mccrack googlemail com>]

On Wed, 2008-08-13 at 17:00 +0100, Martyn Russell wrote:
> Jamie McCracken wrote:
> > On Wed, 2008-08-13 at 17:12 +0200, Carlos Garnacho wrote:
> > > Hi!,
>
> Hi :)
>
> > > On mar, 2008-08-12 at 14:18 -0400, Jamie McCracken wrote:
> > >
> > > <snip>
> > >
> > > > that sounds inefficient - trunk only ever checked for existing deleted
> > > > or junk emails at startup because iterating through all emails in the
> > > > summary files is expensive. 
> > > > From what I've read in trunk code, you still iterate through all the
> > > mails in the summary in check_summary_file(), and you will have to
> > > iterate over them again later to index new messages, etc...
> >
> > yes but when we are not doing the startup check, we are skipping so its
> > faster and we are not stopping at any deleted or junk email and checking
> > it 
>
> Of course it is faster, but that doesn't mean we are completely
> synchronised - unless I missed the point. If you had an email in the
> summary file before and then you mark it as deleted or junk, the summary
> file is out of date. If this is done when tracker isn't running or on
> another machine, etc. you would _HAVE_ to read the summary file again on
> start up to make sure you were synchronised. At least that's how I
> understand it.
>
> > > As far as I know, it's quite unavoidable to parse again summaries, since
> > > under some circumstances Message IDs could be reused, which would leave
> > > you with inconsistent data in the DBs. Even if it isn't, expunging a
> > > folder would render any stored offset for the summary file useless (even
> > > dangerous).
> >
> > true but we would get a deletion from inotify of the summary file if
> > that was the case. Its not a byte offset but message count - so we skip
> > x messages to get the new ones (similar to what beagle does)
>
> As I illustrated above, you can't guarantee Tracker is either:
>
> 1) running all the time
> 2) email isn't deleted/etc from another machine/client/webserver/etc.

such synchronicty can be done at startup - we will know if something is
modified

theres no need to attempt it everytime an email arrives which is my
point

it was optimised before but the changes in your branch have removed this
- pls revert

jamie