Re: [Tracker] Delaying release to end of weekend while I investigate...



2006/8/4, Jamie McCracken <jamiemcc blueyonder co uk>:
Mikkel Kamstrup Erlandsen wrote:
> 2006/8/4, Jamie McCracken <jamiemcc blueyonder co uk
> <mailto:jamiemcc blueyonder co uk >>:
>
>     Senna (http://qwik.jp/senna/mysql_binding_en.html) which can be patched
>     into mysql directly to give better, faster more accurate full text
>     search.
>
>     Should imporve things a lot especially when fulltext indexes and data
>     files are bigger than available memory (which causes the current
>     fulltext search to slow down a *lot* due to the extra disk thrashing)
>
>     The senna indexes are much much *smaller* and should fit in available
>     memory far more easily as a result.
>
>     The ngram stuff should provide more fuzzy based search too.
>
>
>
> This sounds extremely cool. However this means that you must ship a
> patched mysql with tracker, no?

yes thats why I am experimenting with it. I could also patch senna
itself to support stemming (an important feature missing from tracker
and mysql which beagle currently enjoys).

As the senna word index also stores count frequency it also means that
ranked searches will be as fast as unranked ones are.

It would mean inlining a cut down version of mysql source tree into
tracker. As tracker only statically links the mysql stuff none of the
inlined mysql (headers and libs) will get installed on the users machine
so it wont interfere with existing mysql installations.

It also gives us stability against more radical changes in mysql which
are not guaranteed backwards compatible and prevent issues like the
-yassl one that came up on debian sid recently.

Oh and when it gets into Gnome it will need to have the mysql source
inlined anyway so people can build it with jhbuild.

I will need to work on cutting out the cruft from 80mb of the
uncompressed mysql source tarball though (I should be able to get it
down to a more reasonable 15mb by removing superflous stuff that doesn't
get built when you specify just the embedded server)


>
>     As they would require a database rebuild to apply, I think its best I
>     delay the release and play around with the senna patches first.
>
>
> If the benefits you outline above will come out of it, delay as much as
> you like :-)

Time will tell :)


However cool Senna is I can't help but feel that inlining mysql is a BIG deal... As a dev i personally don't mind, but I guess it might give you trouble in the long run...
 
 - How big will the maintenance cost be?
 - Is the 15MB you talk about acceptable? 15 MB is a lot!
 - Could Senna be an optinal dep? Tracker falling back to ordinary embedded mysql if it is not detected?
 - You *will* receive a lot of flak on this issue when you propose for Gnome inclusion if mysql is inlined

Cheers,
Mikkel


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]