Re: Followup: opinions on Search services

[Behdad Esfahbod <behdad cs toronto edu>]

On Tue, 26 Apr 2005, Manuel Amador wrote:

> El jue, 07-04-2005 a las 14:56 -0400, Behdad Esfahbod escribió:
>
> > Lucene is a decent search engine.  You cannot compare it with SQL
> > databases, you can compare it to another search engine, that may
> > or may not use SQL databases as backend, but, as soon as you are
> > talking about search engines, their implementation details
> > doesn't matter at all.  So, SearchServices by using SQL databases
> > is really losing here, since it has to do a lot to catch Lucene,
> > that I doubt it can.
>
> We use the MATCH statement of MySQL.  We really do not need that much
> more =).

So you are bound to MySQL.

> >   SQL databases are good things if you want
> > atomicity, transactions, scalability, support for (really)
> > complicated queries: joins, subqueries, etc.
>
> Which we use heavily.

Errmm,  you said you do not need that much more?  BTW, using
these features is your implementation details.  Desktop search
service does not need these features.  Who cares whether your
search is done atomic?

> >   None of which is
> > needed at all in a Desktop search service that you have one
> > single server per user that does the indexing too.
>
> We have only one server for all users.  We thus dramatically reduce the
> load on multiuser systems.

That design philosophy is dead.  Your system needs root access to
the machine and forces user to run MySQL.  You need to do all
nasty permission handling.  Having one server per user does not
necessarily mean more load.  Processes are cheap in Unix.


> > Lucene on the other hand, comes out from an experience
> > ex-employer of altavista, and from the Apache Foundation.  It's
> > specialized for search services.  It allows for localization of
> > search technology:  You have an English normalizer, a German one,
> > a Persian one, ....  Yes, you have text normalizers there.
>
> That is very good, I grant you that.  But there's no way we could
> intersect Lucene results with our own metadata search results, such as:
>
> - documents created by rudd-o the last month that contain the word
> "paloma" and were written in plain text
>
> ... unless we could move completely to Lucene, which you appear to have
> done.  By the way, does Lucene have pybindings?

Yes, and I highly recommend that.  Lucene does not have Python
bindings, but you can either use jython and live with the JVM, or
use Lupy which is a port of Lucene 1.2 to Python with continued
development.

--behdad
http://behdad.org/