Re: [Tracker] Releasing Tracker 0.6.90

[Jamie McCracken <jamie mccrack googlemail com>]

On Thu, 2009-02-05 at 15:50 +0000, Martyn Russell wrote:
> Jamie McCracken wrote:
> > On Thu, 2009-02-05 at 14:03 +0000, Martyn Russell wrote:
> > > Jamie McCracken wrote:
> > > > Thanks I will check over weekend and try and fix some outstanding issues
> > > > with TST
> > > >
> > > > If all goes well we can release next week
> > > Jamie I was planning on releasing tomorrow. If we keep going on like 
> > > this, we will never release.
> >
> > what about the deletion performance issue?
>
> TJ recently filed a bug for 0.6.6 where deleting emails in the current 
> release brings the computer completely to its knees and you can't kill 
> trackerd either. This is ALREADY much better than it was in the last 
> release. It is better to be slow and responsive than block and be 
> unresponsive.
>
> Carlos was telling me in a discussion we just had, that the kernel has 
> 20k source files - that's *just* .[ch]. There are likely to be other 
> files in there too of course which get indexed. He says this takes 20 
> minutes (maximum to his recollection) to delete. That's almost 400 Mb of 
> pure readable data file to index. Removing that from the databases is 
> going to take some time.
>
> The deletion performance is down to QDBM. This *should* be significantly 
> improved with FTS. We are not doing that before this release either, so 
> the sooner we release, the sooner we can get on with FTS support and 
> really try to fix this issue.

we are not deleting from qdbm! If you have changed stuff to do this then
please revert. O.6.6 never deleted from qdbm and indeed never should as
its prohibitively expensive

we are only deleting from sqlite. Phantom hits are weeded out by
checking if service ID exists in sqlite. Hit counts for searches are
estimates only (same as they are for google) 

jamie