Re: Followup: opinions on Search services

[Jamie McCracken <jamiemcc blueyonder co uk>]

John (J5) Palmieri wrote:
> On Wed, 2005-04-06 at 16:27, Jamie McCracken wrote:
>
> > Joe Shaw wrote:
> >
> > > Hi,
> > >
> > > On Tue, 2005-04-05 at 21:58 -0500, Manuel Amador wrote:
> > >
> > >
> > > > I was wondering, did any of you guys had a shot at trying Search
> > > > services?  Did you find it useful?  Do you have any ideas for
> > > > improvement in this area?
> > >
> > >
> > > I haven't played with it yet, but I did peruse the code a little.  My
> > > question is, why a new project when something like Beagle is aimed at
> > > almost the exact same problem?
> >
> > The problem is that neither of them are really optimal solutions nor are 
> > they likely to achieve broad acceptance for one reason or another.
> >
> > However for technical reasons :
> >
> > 1) Indexing is both CPU and resource intensive ergo Python and C# are 
> > not really a good choice for this - it would be a lot better to do it in C.
>
>
> This is a misnomer.  C code can be every bit as inefficient and where
> Python and C# are inefficient one can always go down and write modules
> in a lower level language.  Point is don't optimize until you need to.

Actually mixing unmanaged with managed code is a real performance killer 
for mono. ANyway, the point was more to do with memory usage and in 
particular how garbage collected languages are currently quite poor for 
implementing this - see the beagle webpage about memory usage and its 
problems in this instance.

> > 2) Use of an SQL database is a far superior, faster and flexible 
> > solution to using a dedicated indexer like the lucerne engine (all other 
> > competing engines like spotlight use sql databases). This is one area 
> > search services has got right.
>
>
> Don't know much about lucerne but a dedicated solution can often be
> faster than a general solution such as a database.  Again I think this
> is another misnomer without hard data to back it up.

Yes Dedicated is better but we are dealing with metadata + indexed data 
which favours a relational DB solution for the extra flexibility. There 
are pros and cons for this of course but the dedicated solution is not 
as expandable. EG say i want my indexer to auto thumbail all images and 
video files as they are created so Nautilus is not slowed down by 
thumbnailing - thats pretty easy to do with a DB but for a text based 
lucerne engine I have no idea how they could do that efficiently.

> > 3) It really needs to be a freedesktop solution written in C cause it 
> > looks likely that KDE will produce their own solution so adding to a lot 
> > of needless wheel reinvention.
>
>
> Freedesktop is not a dumping ground for every little bit of technology
> that seems interesting.  Without divergence there is no diversity or
> competition.  Your instance on C is an example of stagnation that could
> cause projects to never get off the ground.  Things that need to be on
> freedesktop.org eventually get there.  I fear that people are using it
> as a holy grail that somehow legitimizes projects and magically makes
> them acceptable to all parties.

My understanding is that freedesktop is designed to improve 
interoperability and prevent needless reinvention of the wheel across 
desktops. I dont see how having multiple solutions to this problem is of 
benefit to anyone. If competition is required we already have it in OS/X 
spotlight and longhorn.

jamie.

> --
> J5