Re: [Tracker] Delaying release to end of weekend while I investigate...



Mikkel Kamstrup Erlandsen wrote:
2006/8/4, Jamie McCracken <jamiemcc blueyonder co uk <mailto:jamiemcc blueyonder co uk>>:

    Senna (http://qwik.jp/senna/mysql_binding_en.html) which can be patched
    into mysql directly to give better, faster more accurate full text
    search.

    Should imporve things a lot especially when fulltext indexes and data
    files are bigger than available memory (which causes the current
    fulltext search to slow down a *lot* due to the extra disk thrashing)

    The senna indexes are much much *smaller* and should fit in available
    memory far more easily as a result.

    The ngram stuff should provide more fuzzy based search too.



This sounds extremely cool. However this means that you must ship a patched mysql with tracker, no?

yes thats why I am experimenting with it. I could also patch senna itself to support stemming (an important feature missing from tracker and mysql which beagle currently enjoys).

As the senna word index also stores count frequency it also means that ranked searches will be as fast as unranked ones are.

It would mean inlining a cut down version of mysql source tree into tracker. As tracker only statically links the mysql stuff none of the inlined mysql (headers and libs) will get installed on the users machine so it wont interfere with existing mysql installations.

It also gives us stability against more radical changes in mysql which are not guaranteed backwards compatible and prevent issues like the -yassl one that came up on debian sid recently.

Oh and when it gets into Gnome it will need to have the mysql source inlined anyway so people can build it with jhbuild.

I will need to work on cutting out the cruft from 80mb of the uncompressed mysql source tarball though (I should be able to get it down to a more reasonable 15mb by removing superflous stuff that doesn't get built when you specify just the embedded server)



    As they would require a database rebuild to apply, I think its best I
delay the release and play around with the senna patches first.

If the benefits you outline above will come out of it, delay as much as you like :-)

Time will tell :)



--
Mr Jamie McCracken
http://jamiemcc.livejournal.com/




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]