Re: [Tracker] Tracker daemon/indexer responsibilities

[Jamie McCracken <jamie mccrack googlemail com>]

On Thu, 2008-06-26 at 17:26 +0200, Carlos Garnacho wrote:
> Hi!,
>
> On miÃ, 2008-06-25 at 12:16 -0400, Jamie McCracken wrote:
>
> <snip>
> > Not quite what I had in mind - the indexer should be dumb and fed stuff
> > to index by the daemon. the exception is directories which need to be
> > recursively scanned (not sure we need separate queues for them)
>
> I did it this way because 1) it's fully deterministic and 2) we
> guarantee the queues are close to the smallest possible. If things were
> in the same queue and you had a dir with say 100 subdirs and 200 files,
> there's a good chance that plenty of those subdirs are expanded (and the
> contained files pushed in the queue tail) before the files are
> processed, which could make the queue grow considerably.
>
> Having a separate queue for directories makes the files queue just grow
> up the necessary to contain the files in the last processed directory,
> which seems to be quite memory friendly :)

yeah makes sense - thats what we currently do with dir_list

jamie