Re: [Tracker] Tracker -.0.6.3 version needs testing prior to release

[jamie <jamiemcc blueyonder co uk>]

On Mon, 2007-09-24 at 01:45 +0200, Laurent Aguerreche wrote:
> Le dimanche 23 septembre 2007 Ã 14:12 +0100, jamie a Ãcrit :
> > Hi All,
> >
> > the next version of tracker is sitting in svn (there are still a few bug
> > fixes to apply but should be stable for the most part)
> >
> > this version implements index merging which dramatically improves speed
> > and scalability of indexing (as well as eliminating fragmentation and
> > disk IO issues0. 250 MB of linux kernel source was indexed in less than
> > 8 minutes so it should index large amounts in very short time)
> >
> > because of that there are some major changes in the way things ar e
> > handled so it really needs a lot of thorough testing before release
> >
> > we use by default a 16MB word cache as each flush produces a separate
> > mini-index which are later all merged together to form the main index.
> > Search results will only be available after the first flush and no new
> > search results will be added from subsequent flushes until merging is
> > complete
> >
> > so please can you test the following:
> >
> > 1) index as normal (no need to use --reindex - it should reindex itself
> > auotmatically)
>
> My previous experience with trackerd going around 180Mo tends to say
> that there could be a problem there... More testing needed.
>
> > 2) keep an eye on memory usage - there may be a few leaks but it should
> > plateau around 30mb
>
> It is more 50Mo or something... Did you introduce Mono, Java or Python
> somewhere in trackerd?  ;-)

im fixing this - email indexing causes the most memory usage!

> > 3) after index test makking changes to files and moving stuff around
>
> I moved ~440Mo of code sources/html from one directory to another and
> trackerd stayed stuck with CPU 100% several seconds (I had to kill
> it...).

thats bound to be slow moving around that many files!

if it stayed stuck for a few secs I would say its ok


> > 4) do source checkouts so that lots of files are overwritten in one go
> > and pay attention to speed/slowdown of system
>
> It seems to be better than before but I still have emacs freezing
> sometimes when I want to save a file.

the freezes should be less than a few secs?

also can you tell if freezes occur when indexing files or emails? (i
suspect emails) or are confined to merging (where heaviest disk IO
occurs)

> > 5) create lots of new files and make sure its fast and new results show
> > up
>
> New files are not shown very fast...

thats ok so long as they are shown and indexing of these new files is
fast


> > pls also run with -f in gdb ss thta you can get me backtraces for
> > non-crash errors (as well as crashed ones!)
> >
> >
> > if all goes well I will release tomorrow...
>
> I think we should not!

well we will see tomorrow night!