Re: [Tracker] Sqlite now default in cvs (mysql backend wont compile yet!)

[Jamie McCracken <jamiemcc blueyonder co uk>]

Eyal Oren wrote:
> On 09/26/06/09/06 02:14 +0100, Jamie McCracken wrote:
>
> > 1) differential indexing for faster updates
> > 2) stemming so searches for "penguin" will match against "penguins"
> > 3) Bzip compression of full text so it has much smaller footprint in the 
> > sqlite db
> again, just out of curiosity, why do we store th (compressed) full-text if 
> we have the inverted index? so that we can -for each match- show snippets 
> of text around the matched keywords (context of the match)?

yes + also we need the unique word list to do updates and deletes to the 
inverted index

I could store only the unique words + score instead of the full text for 
text files but they wouldn't be much smaller than the zippped contents 
and we also need the full text to do exact phrase searches.

--
Mr Jamie McCracken
http://jamiemcc.livejournal.com/