Re: State of the Pooch

[Joe Shaw <joeshaw novell com>]

Hi,

On Thu, 2006-11-16 at 16:33 -0500, Debajyoti Bera wrote:
> The memory reduction and speed improvement are _huge_. I decided to switch CVS 
> to generics a couple of week ago but some runtime crashes during my testing 
> with 1.1.18 stopped me. Unfortunately, 1.2 is not suitable for the extended 
> attribute problem Joe mentioned in an earlier problem. But the switch is 
> happening soon.

I've been running with generics on the branch since I created it and
haven't had any runtime crashes.  If you see any, please file bugs at
http://bugzilla.ximian.com.

I'm hoping there is some low-hanging fruit in Lucene that we can convert
to generics to take off a large chunk of memory.

> The infrastructure is mostly there. One of the reasons I feel none of the 
> backend authors (ahem) use it because none of the known UIs actually show 
> this information. GUI people please figure out how to inform the user about 
> indexing and index status and the backends will get straightened out.

The infrastructure is sorta there.  There's no way to say "this is the
initial, first time crawl", for example, which is what most people are
actually interested in.

I think it'd probably be a good idea to add a fairly generic message
type so that the daemon can send informational messages to the clients.
(I'm doing the initial index, I'm finished with the initial index, etc.)

> > * Handling crashes in the index helper better.
> 
> This is a killer. Its on my cards once unified index lands. Since there wont 
> be a separate index for each backend, a single bad file could potentially 
> ruin data from other backends. The webbrowser backends e.g. wont be able to 
> recreate the index if purged. They need to be in a separate index.

Yeah, I'm not too worried about the index being corrupted, but we need
to be smarter about the receipts that the index helper returns to the
daemon.  When a crash occurs, we should try each of the items in the
crashing batch individually to narrow down the exact file that crashes,
and then mark it (probably with an xattr) to not try to filter it ever
again.

> Can we put the libbeagle documentation on the wiki ? Like the glib docs. For 
> some reason I cant generate the documentation on my machine and having an 
> updated API documentation on the web will make people interested in using the 
> API. I can do it, but how do I add a bunch of html pages to the wiki as-is ?

I can look into doing this.  I'm not sure we'll be able to automate it,
and we'll want to keep it outside of the wiki structure, I think.

Joe