Re: [Tracker] Delaying release to end of weekend while I investigate...

[Jamie McCracken <jamiemcc blueyonder co uk>]

Mikkel Kamstrup Erlandsen wrote:
> 2006/8/4, Jamie McCracken <jamiemcc blueyonder co uk 
> <mailto:jamiemcc blueyonder co uk>>:
>
>     Senna (http://qwik.jp/senna/mysql_binding_en.html) which can be patched
>     into mysql directly to give better, faster more accurate full text
>     search.
>
>     Should imporve things a lot especially when fulltext indexes and data
>     files are bigger than available memory (which causes the current
>     fulltext search to slow down a *lot* due to the extra disk thrashing)
>
>     The senna indexes are much much *smaller* and should fit in available
>     memory far more easily as a result.
>
>     The ngram stuff should provide more fuzzy based search too.
>
>
>
> This sounds extremely cool. However this means that you must ship a 
> patched mysql with tracker, no? 

yes thats why I am experimenting with it. I could also patch senna 
itself to support stemming (an important feature missing from tracker 
and mysql which beagle currently enjoys).

As the senna word index also stores count frequency it also means that 
ranked searches will be as fast as unranked ones are.

It would mean inlining a cut down version of mysql source tree into 
tracker. As tracker only statically links the mysql stuff none of the 
inlined mysql (headers and libs) will get installed on the users machine 
so it wont interfere with existing mysql installations.

It also gives us stability against more radical changes in mysql which 
are not guaranteed backwards compatible and prevent issues like the 
-yassl one that came up on debian sid recently.

Oh and when it gets into Gnome it will need to have the mysql source 
inlined anyway so people can build it with jhbuild.

I will need to work on cutting out the cruft from 80mb of the 
uncompressed mysql source tarball though (I should be able to get it 
down to a more reasonable 15mb by removing superflous stuff that doesn't 
get built when you specify just the embedded server)


>     As they would require a database rebuild to apply, I think its best I
>     delay the release and play around with the senna patches first. 
>
>
> If the benefits you outline above will come out of it, delay as much as 
> you like :-)

Time will tell :)



--
Mr Jamie McCracken
http://jamiemcc.livejournal.com/