Re: [Tracker] Releasing Tracker 0.6.90

[Martyn Russell <martyn imendio com>]

Jamie McCracken wrote:
> On Thu, 2009-02-05 at 14:03 +0000, Martyn Russell wrote:
> > Jamie McCracken wrote:
> > > Thanks I will check over weekend and try and fix some outstanding issues
> > > with TST
> > >
> > > If all goes well we can release next week
> > Jamie I was planning on releasing tomorrow. If we keep going on like 
> > this, we will never release.
>
> what about the deletion performance issue?

TJ recently filed a bug for 0.6.6 where deleting emails in the current 
release brings the computer completely to its knees and you can't kill 
trackerd either. This is ALREADY much better than it was in the last 
release. It is better to be slow and responsive than block and be 
unresponsive.

Carlos was telling me in a discussion we just had, that the kernel has 
20k source files - that's *just* .[ch]. There are likely to be other 
files in there too of course which get indexed. He says this takes 20 
minutes (maximum to his recollection) to delete. That's almost 400 Mb of 
pure readable data file to index. Removing that from the databases is 
going to take some time.

The deletion performance is down to QDBM. This *should* be significantly 
improved with FTS. We are not doing that before this release either, so 
the sooner we release, the sooner we can get on with FTS support and 
really try to fix this issue.

I would be interested to know if doing the same test on the 0.6.6 
release would be any quicker at all. IT most probably would be because 
it wasn't actually removing words from the QDBM database for deleted 
words. Not only that, but IIRC, it didn't recursively delete items in 
the database either (at least, I remember adding the SQL to do that).

> Im quite happy to wait another week or two to fix up a potentially
> unpleasant experience

There are still likely to be other unpleasant experiences left in there 
we don't know about. But we have been in testing now for over 2 months 
and unlike in the early days when we started the indexer split branch, 
the guys in the team aren't beating down our doors for Carlos and I to 
fix crashers and blockers in the daemon/indexer. So I consider the 
experience to be quite pleasant even if the performance can still be 
improved in areas.

We have also been waiting over a month now to release with these little 
things that keep cropping up. Releases should be much more often.

--
Regards,
Martyn