Re: Call to test CVS

[Joe Shaw <joeshaw novell com>]

Hi,

On Sat, 2006-12-09 at 04:02 +0100, Stephan Hegel wrote:
> Wouldn't it be better to look at the size of the archive (first estimate)
> and/or the size of the files within the archive ? The archive which wasn't
> indexed is less than 800k. It is very unlikely that it exhausts my disk
> space when extracting.

Yeah, definitely this would be a better thing to do.  There is also an
architectural issue here, though.  The more files that are extracted
from the archive, the more this will affect memory usage in the
daemon(*).  So extracting 200 small files is also not a good idea.  I'll
look into making it size-based, but we may have to go with the
admittedly arbitrary hard limit for now.

Joe

[1] The technical details: right now filtering happens in the index
helper process.  Inside of filters is where child indexables are
generated.  When the filtering has finished, the index helper sends back
"receipts" to the main daemon indicating that a bunch of child
indexables have been created.  Those are processed by the daemon and
sent back to the index helper.  What we should probably do instead is
(a) not generate all the indexable objects up front if we can avoid it,
and (b) don't bother to send child indexables back to the daemon -- we
should probably just handle them entirely within the helper.