Index Helper for beagle++ hangs occasionally

[Max Wiehle <max wiehle gmail com>]

Hi.
I've been hacking a bit on beagle++ lately.
And i ran into a strange thing i could not figure out.

We are using external programms to do some of the metadata extraction
for us. The communication works via stdin and stdout - i.e. we start the
programm from within the IndexHelper and then read and parse its stdout.
Now from time to time IndexHelper will stop reading even though more
lines are available on stdin. It sits in a 
	while((line = StreamReader.ReadLine()) != null)
loop and does nothing.
The whole thing happens during the flush. All flushed filenames are
handed to the external filter via its stdIn and then the metadata is
read from its stdout.

Any idea what might be causing such a behaviour?
Here's what my log says - look at the timestamps.

20071208 03:12:24.1883 08549 IndexH DEBUG: read line:
<email://local local/Sent;uid=542/mwiehle2%40ix.urz.uni-heidelberg.de>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
<http://www.semanticdesktop.org/ontologies/2007/03/22/nco#PersonContact> .

20071208 03:12:25.9373 08549 IndexH DEBUG: Helper Size: VmRSS=22,1 MB,
size=1,80, 20,0%

20071208 03:31:30.4364 08549 IndexH DEBUG: read line:
<mailto:mwiehle2 ix urz uni-heidelberg de>
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
<http://www.semanticdesktop.org/ontologies/2007/03/22/nco#EmailAddress> .

All the lines have been printed almost imideatly after each other and i
verified that by tee'ing the external filters output to a file. 

I don't have any idea what keeps the Index-Helper from Reading this for
almost 20 minutes. So i wonder if it's something specific about the
Index-Helper especially since the Helper Size message pops up inbetween.

Looks like BEAGLE_EXERCISE_THE_DOG was not set but i still wonder why it
would sleep for 20 minutes especialy since the machiene was not under
heavy load.

Thanks for any hint in advance.
 Max