Re: [Evolution-hackers] ... and how camel should be

[Philip Van Hoof <spam pvanhoof be>]

On Tue, 2006-02-14 at 14:03 -0500, Lee Revell wrote:

> > > I really don't think the message IDs are the main source of bloat in
> > > Evo.
> > 
> > I measured it ;-)
> > 
> > 10.000 E-mails used +-8MB of memory.
> 
> Do you think that's a problem?

For evolution, it might be a lesser problem than for camel. I'd like to
start using camel on mobile devices.


And more importantly, http://bugzilla.gnome.org/show_bug.cgi?id=331017
not only describes the problem about camel_folder_get_uids as a function
that returns a lot memory for large folders.

The more important issue is that it can't request the headers in a given
sort order. This makes it necessary to, when sorting a view, read every
single message header (the gtktreesortable stuff will do this behind
your back). Also the ones that weren't loaded yet.

Now read below, my theory about not having to load everything in memory
which I proved using tinymail as prove of concept. Please check it out
before thinking the theory is just a theory: I "did" successfully
implement this. It "does" work correctly.

So in case you sort your view, you do have to read everything in memory
just to read the message headers. 

A camel_folder_get_uids that can return a sorted list wouldn't require
me to do that.


The theory of not having to load the CamelMessageInfo instances:

I didn't add the CamelMessageInfo instances to the count. Those will add
a significant number to the total amount of used memory. Far more than
the uids I, as those headers are a lot longer in total length than the
uids.

However .. that can be fixed if evolution would call camel_message_
info_free on all the message_info instances that become invisible.

I quickly studied evolution by searching for invocations on this _free
method, and it doesn't look like evolution is doing that.


A prove of concept of this theory can be found here:

https://svn.cronos.be/svn/tinymail/trunk/libtinymail-camel/tny-msg-header.c
https://svn.cronos.be/svn/tinymail/trunk/libtinymailui/tny-msg-header-list-model.c

Check the methods tny_msg_header_list_model_unref_node and
tny_msg_header_uncache in those files. Feel free to tryout tinymail. I
"did" implement the theory and it "does" work. And I "did" measure it.
You can see the measurement happen as I added some debugging printfs
that will tell you when it's allocating and when it's deallocating a
CamelMessageInfo instance.

Try scrolling the view, you'll notice that it "does" both allocate and
deallocate. That's corresponding rows becoming visible and invisible.

...

That's CamelMessageInfo getting therefore allocated and deallocated.

Using my custom tree-model, I have very tight control over what I really
really need. And what I don't really need.

--> I don't need the invisible message headers. I simply don't.


The fact that evolution does, IS probably a serious memory problem. I'll
spare you guys the measurement. For certain (large) folders, I fear it's
going to be huge.


> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> rlrevell 11659  5.4 37.0 260516 162700 ?       Sl   Feb13  66:07 evolution --component=mail

> Even if you're off by 4x, that's only 32MB of a 162MB footprint.  Surely
> we waste a lot more on the GUI...

You are measuring totally wrong. "ps aux" will not give you a correct
picture of the used memory, as a lot mmap'ed shared libraries are added
to the count. Please don't use it.

> Personally I think Evo is already slow enough, and we cannot afford to
> trade off any speed to save memory.

So we just load everything in memory?! Why not also load all the e-mails
of all your folders into memory!! That will speedup evolution! 

			No it wont.

But it will slowdown the rest of your computer.


-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be