Re: Current TODO

["D Bera" <dbera web gmail com>]

Lucene search is actually quite fast. A little bit of speed is lost
since beagle has to merge queries from its primary and seconary
indexes and do some actual filesystem traversal to (1) find out the
actual paths of the results, (2) find out if a hit is valid; but
nevertheless it is quite fast. E.g. using the python bindings, I
generally get back search results in ~ 0.1 seconds.
beagle-search seems to be slower since it has to do a lot more to
present the results in a friendly manner, which includes (1)start
fetching snippets in background, (2) handle thumbnails and icons and
some more stuff.
If you want to see near real-time query with beagle, I suggest you try
out deskbar.

- d.

> Any other thoughts on this? I think the metadata only query to start
> seems like a good idea, but I'm not really familiar with the
> intricacies of the query system for beagle, I more understand the
> indexing.
>
> Anyways, assuming lucene doesn't automatically check metadata first,
> maybe building a index or something that just contains the title and
> keywords is a good idea. Query that and return those results first,
> then do the fulltext search, I'm gonna start rooting around the query
> code, but just wanted to throw that thought out there.
>
> -Kevin Kubasik
>
> On 2/15/06, Kevin Kubasik <kevin kubasik net> wrote:
> > I refer more to the beagle-query --livequery from the command line, I
> > did some rough (and generally poor) benchmarking attempts with that,
> > once I can get some sort of reliable/standardized benchmark, I'll
> > report those numbers. I'll see if I can't get together some sort of
> > screencast for GDS or Spotlight to demonstrate better. Its more in the
> > responsiveness, even on lowerend systems. I know that Spotlight
> > accomplishes a large chunk of this through distributed indexes across
> > the disk, and searching them based on modification time (more recently
> > modified files searched first), but thats based on an old forum posts,
> > I've linked to a few whitepapers on Spotlight, they don't seem to have
> > much on the superinteresting side, and most feel highly
> > over-simplified, but maybe they can help.
> >
> > This one is quite good, a large part of the speed is based on
> > searching a simple metadata store and returning those results before
> > handling the fulltext index.
> > http://developer.apple.com/macosx/spotlight.html
> >
> > An API reference to Apple's 'Search Kit' I started to root around for
> > some hints at how performance was increased so substantially (Apple
> > claims a 3x speedup over its last release)
> > http://developer.apple.com/documentation/UserExperience/Reference/SearchKit/
> >
> > Oddly enough theres not much on the innards of Googles search, solid
> > and almost too-good-to-be-true API docs, but little mention of the
> > beast beneath.
> >
> > I plan on checking yahoo for some developer info on there desktop
> > search, but it really hasn't been popularized much, so I can't say I
> > know much about it, but they have been great about sharing info/code.
> >
> > Cheers,
> > Kevin Kubasik
> >
> >
> > On 2/15/06, D Bera <dbera web gmail com> wrote:
> > > > I dunno, search engine optimization is not something I have much
> > > > experience in, but one of the major blockers for beagle when compared
> > > > to search applications such as GDS and Spotlight is the speed of a
> > > > partial query. While a complete query is quick, the first fragments of
> > > > a word (such as fragm from fragment) are not very speedy, just a
> > > > thought to maybe inspire some interesting discussion.
> > >
> > > I doubt there is "search-engine" optimisation in the usual database
> > > sense, but there are scopes of improvement. Your comment above is
> > > interesting and will be more so, if you can provide some rough
> > > numbers/statistics. I dont have access to GDS or spotlight, hence if
> > > someone posts some comparison figures, it will be helpful.
> > > Also, when you say "the first fragments of a word (such as fragm from
> > > fragment) are not very speedy...", how did you evaluate that in beagle
> > > ? lucene allows wildcard query or term enumeration which can be used
> > > for presenting matches for partial text, but none of this is currently
> > > implemented in beagle (afaik) ?
> > >
> > > - d.
> > >
> > > --
> > > -----------------------------------------------------
> > > Debajyoti Bera @ http://dbera.blogspot.com
> > > beagle / KDE fan
> > > Mandriva / Inspiron-1100 user
> > >
> >
> >
> > --
> > Cheers,
> > Kevin Kubasik
> >
> > http://kubasik.net/blog
> >
>
>
> --
> Cheers,
> Kevin Kubasik
>
> http://kubasik.net/blog
>


--
-----------------------------------------------------
Debajyoti Bera @ http://dbera.blogspot.com
beagle / KDE fan
Mandriva / Inspiron-1100 user