Re: [Tracker] more issues with indexer-split

[Jamie McCracken <jamie mccrack googlemail com>]

On Fri, 2008-08-15 at 11:33 +0100, Martyn Russell wrote:
> Jamie McCracken wrote:
> > On Wed, 2008-08-13 at 19:30 +0200, Carlos Garnacho wrote:
> > > As far as I see, for mbox you're storing the offset in the stream:
> > >
> > >         msg_offset = g_mime_parser_tell (mf->parser);
> > >         ....
> > >         mail_msg->offset = msg_offset;
> > >         
> > > For IMAP, I just get "0" in the Services table, also didn't get to see
> > > any code to do this.
> >
> >
> > imap stores message count too - its count rather than byte offset

> As Carlos says, this code is NOT working in TRUNK for IMAP. So this
> whole argument is moot.

ï
i will look into this for imap. Mbox/pop3 definitely does work as I
explained


> > > > no when junk/deleted email is encountered during the start up scan its
> > > > UID is checked against that table  (JunkMails) to see if we already know
> > > > about it. If its not in that table then we add it and then delete it
> > > > from our index. Ergo its more efficient than what you have
>
> The whole idea of keeping a separate table for deleted/junk email sounds
> really inefficient to me. I have quite a bit and I get quite a bit every
> day, that's a lot of extra processing. Surely it is MORE processing than
> the current inefficiencies you are outlining with our current design?

no because in most cases the junk emails table will be smaller than the
email services table 



> > > Could you tell me where's that code? The only users for
> > > InsertJunk/LookupJunk (the stored procedures) are
> > > tracker_db_email_insert_junk() and tracker_db_email_lookup_junk(), the
> > > former is also the only user of the latter, and it doesn't do what you
> > > mention.
> > >
> > > The only place I see where it could delete emails from the DB for
> > > Evolution is check_summary_file(), and tracker_db_email_delete_email()
> > > seems to be called inconditionally for any junk/deleted message found.
> >
> > the way it should work is as described above
> >
> > I had tested it and it works (deleted and junk emails are pruned on next
> > restart of trackerd)
>
> What you're saying here doesn't make a lot of sense to me. It sounds
> like you're saying that if mail is marked as junk or deleted you don't
> want to update the index until we restart the daemon? So people will
> still be searching and finding junk until trackerd is restarted? That
> doesn't sound right to me. Or did you mean something else?

thats right if a user marks an email in evo as junk or deleted it will
remain in tracker until tracker is restarted - that is deliberate 

as I said before we dont wanna do junk checking everytime a new email
arrives which is crazily inefficient hence we skip that part til
trackerd is restarted

anyway iw ill optimize it - can you sort out the file moves and restore
inotify api if necessary. I will review what you have done over the
weekend and hopefully merge monday

jamie