Re: Gnome Journal Article and General Beagle Publicity

[Kevin Kubasik <kevin kubasik net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256



D Bera wrote:
> Hi
> 
>> beagle in Foresight for well over a year now.  The great news is for
>> > quite a while now there haven't been any users asking "What is this
>> > beagled thing and why does it use all my memory?".
> 
> Only if you dont do the rounds of distro bugzilla and newbie forums.
> Those places are full of beagle eating CPU and memory problems (some
> of them are due to some old buggy version of beagle that hasnt been
> updated in the distros). I dont think beagle is newbie ready i.e.
> plug-and-play yet.
> 
>> Especially with the most recent release, and the upcoming changes, which
>> is why I think its worth talking about. Using some of the practice
>> mentioned here:
>> http://beagle-project.org/Thunderbird
>> I run beagle (with the thunderbird backend and about 10.000 mails) at
>> about 45 MB resident for the master process and
>> Debug: Helper Size: VmRSS=34.2 MB, size=3.55, 63.7
>> its not perfect, but its certainly far more passive then before, and
>> without the tbird backend, its significantly better.
> 
> Kevin, I had a brief glance at the thunderbird backend. I didnt see
> any obvious place which takes a lot of memory. 
The method Beagle.Util.Mork.Db.Read() Loads the entire database into a
hashtable, with multiple regular expressions per entry, all kept in
memory. Large mork files can bloat beagle to gigs of memory. Check out
http://bugzilla.gnome.org/show_bug.cgi?id=355549
for more info. However, on that note, I have tried some MMaping
solutions (no success as of yet) and some messing with the regexps so
that we can increment the mork reads. (again, no luck), But I will check
out your StreamAsArray suggestion!

- -Kevin

> If the concern is the
> ReadToEnd() in the mork parser, then you can try to replace the huge
> content string with a StreamAsArray type implementation. I replaced
> one ReadToEnd() in the html filter in a similar way, check
> Filters/HtmlAgilityPack/HtmlDocument.cs - it contains the
> implementation. Might be useful.
> 
> - dBera
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iQIVAwUBRVgPdP3xZFNDM330AQgqFxAAiX7QQiVM6uIllsB3GPaZeWMksKT6s5GE
2nfEHoOlvZmyPqfC0+5gVv+wlr0EL76hzILLTC4w4SQfQYeIMLerXPuIkawaecCS
enYpO5y/4/CNqqAicRLTfLkp+gKK5iK1Wn7Z16s1Teovt3miSt5f+PrW+rxPvdWs
Nt/bkCa9XosRIwjLNido+o4a7KfvhNPgGro4AvhyxG3AU0pw2seFAozoQkpzBonR
UY7BtrjgnmaKoo6CJv0PEnQgWl1llqvFA8/sW050w/M1DS3EJaR05lmHc3yTmsXh
ALx46gfgxcgWCspCiH90Nhg+r0R4uBPk0fwpsvSJ6WwKGxq4cqjwPpxsHkWY5x9I
QA9RnFBLAXhxiNbv7UM29u19UbQYeNtefZHFSe6Ustw9m6T5a8KSjU5rH8Q59z4j
sXSEr8AsbXnQnhzHIp7sxLNPHTQL44uNOEtG+fSS3MOO/tzqkpT5W5wtRcqF2K14
f+T31+xE1Xn19euKdqw3obOO1T8REHtCSHGn7FDkd+LqzFd5fHgy6oRvfs+ATDcb
3yUWGsqHBReCzgjLlnkyPT+tVfD9uy9tctFt+lq8STpJY8gURSKCiRM43x2r7XvE
Fx3QaP7FZNSlD94kZG7VciHnB89yfe/MyVRGJscjS/tBDKFki0XozIx6AuGeSCHc
1GGtzpJm67w=
=Qgd/
-----END PGP SIGNATURE-----