Re: [Evolution-hackers] Folder summaries with mmap() (version nine -- fixes a leak)

[Philip Van Hoof <spam pvanhoof be>]

On Tue, 2006-07-11 at 23:14 +0530, Harish Krishnaswamy wrote:

I'm removing the patches mailing list ;-)

> On Tue, 2006-07-11 at 16:47 +0200, Philip Van Hoof wrote:
> > Currently it's obviously consuming more memory when new messages arrive
> > (compared to the mmap technique). In Evolution, such a folder never gets
> > closed. So it will for ever keep using this memory structure (in stead
> > of the information using the mmap).

> I expect the patch to improve things as they stand now and will run it
> through some testing to get the hard numbers. 
> Just curious to know if you whetted this change with some heavy-duty
> search folders at your end. They seem to skew the observations thus far.

The hardest test that I did with it was sorting huge folders using the
Subject and From fields. I didn't yet get Evolution stable enough to do
even more aggressive tests like, indeed, making huge vfolders and
searches.

But after seeing the sorting results, I'm almost confident it will
probably be as fast as in-real-memory. 

The big difference with huge searches (over all folders) will be that
all summary files will be hit. So it's basically a "does mmap scale to
folder-count amount of mmap's?"-question.

The kernel, by default, pages 16 pages ahead of time. You can increase
this using posix_madvise (which isn't platform independent, in case tml
is listening). In random mode, it will not page ahead of time at all.
This will be a nice option for mobile devices with very very very few
memory (but very very very slow sorting and searching).


ps. I think we should do a in-depth tech-meeting about this stuff. What
do you think, Harish? I have a few ideas on reducing memory usage when
Evolution is online (if Evolution is online, summary information of new
messages is still stored in memory, and not immediately reloaded using
mmap) --> this makes Evolution consume (a lot) more memory when its
online, when using the patch (as much as the current implementation).



-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be