Re: Flock Backend

[D Bera <dbera web gmail com>]

> I might be getting a little ahead of myself here, but I wanted to kick
> this idea around a bit. Flock [www.flock.com] is basically an uber-hyped
> up version of firefox with web 2.0 goodness that is out through the
> roof. Now, if we were to expand the indexing of our web history stuff,
> it would seem that opera would be the next choice, but flock stores a
> lucene index of every page visited using CLucene....
>
> Now, I dunno a lot about the specifics of lucene compatibility, but it
> would seem to me, that with maybe a little coordination from the flock
> folks (they have been nothing but extraordinarily friendly and helpful
> so far) we could query that index directly for easy (and basically free)
> access to a complete web history index.
>
> Just getting the idea out there. This is kinda outside my experience,
> since it would not be a traditional backend by any means, but its a
> thought. I see the biggest issue at the moment being a cost-benefit
> thing, since flock can't exactly claim a huge marketshare at the moment,
> but it seems like this might actually be something not too impossible.

Hi Kevin,
  I started using flock since a few yeeks and I feel it holds some promise. I 
didnt know it had a web-history index. If it already stores a webhist index, 
then that would be really cool.
  For the technical part, beagle can already query 3-party database on the fly 
while computing queries (known as QueryDriver - see GoogleDriver in CVS or 
AmarokDriver in bugzilla). If the query to the database is slow, then that 
would be a bottleneck. If flock stores its index as a lucene index, then 
beagle should be able to query it without any significance performance 
overhead. It wont be hard to write a FlockQueryDriver; it would be pretty 
simple too - take a beagle Query, change it to a normal lucene query based on 
the flock index structure and query the flock lucene index (beagle comes with 
a copy of DotLucene). A few things to keep in mind:
* beagle uses (Dot)Lucene-1.9. Make sure flock is using (C)Lucene-1.9 too or 
1.9 is backward compatible with 1.4
* What fields are stored in flock index ? When a beagle query is passed to 
FlockDriver, the (property) fields need to be appropriately modified to Flock 
index. Similarly, when the results come back from FlockDriver, the fields in 
the results need to be appropriately mapped to beagle index fields. The 
time-penalty for this mapping-remapping should not be that high.
* Lucene locks are supposed to be process safe. It shouldnt be a problem if 
both beagle and flock is trying to access the same lucene index.

I cant think of any more off the top of my head. (Oh, one more. IANAL, are 
there licensing issues ??? The only scary thing of "open"-source world.) It 
definitely sounds like a good idea.

- dBera

-- 
--------------------------------------------------------
Debajyoti Bera @ http://dbera.blogspot.com
beagle / KDE fan
Mandriva / Inspiron-1100 user