Re: [Evolution] Evo memory use

[David Woodhouse <dwmw2 infradead org>]

(oooh cute -- evo 1.2 found this even after evo 1.3 segfaulted while I
was typing it)

On Tue, 2003-02-18 at 07:30 in some random timezone that Evo doesn't
want to divulge, Not Zed wrote: 
> On Fri, 2003-02-14 at 20:54, David Woodhouse wrote:
> > Note that when Evolution is separated from the IMAP server by a slow WAN
> > link, the mere fact that it thinks it's necessary to fetch the headers
> > for _every_ message even in the mailbox(es) you currently have open is a
> > bug far more severe than the memory usage.
>
> Hmm, i'm not sure what you mean here.  It only fetches the headers that
> are new.  Or if the server is changing the UIDVALIDITY field, in which
> case we must refetch all headers (as per rfc2060).

But RFC2060 doesn't say you must fetch all headers. It says you must
discard the ones you have. You don't _need_ to know all the headers
unless you're doing local sorting, or unless the user is grabbing the
scrollbar and moving slowly down the folder index, making you actually
display them all.

But you know what I mean because you address it below -- maybe I didn't
make it clear that the two paragraphs were referring to the same thing;
sorry.

> > You know how many messages there are in the mailbox so you can make your
> > scrollbar scale correctly. You can fill in the 'unknown' entries in the 
> > list with some suitable placeholder text and replace that text with the
> > _actual_ message information if and when the relevant list entries are
> > actually displayed on-screen. You can even fetch the message headers for
> > the non-displayed messages in the background to populate your cache, as
> > long as there's no user-driver IMAP activity which should take
> > precedence -- but there's absolutely no excuse for making the user wait
> > while you download all the headers, before you even start to display the
> > list.
>
> Actually there's a couple of reasons :)

Useful information -- thanks. 

>  - updating etable has been extremely slow, and very buggy, we tried
> some incremental update for local mail, and that either caused etable to
> crash a lot, or if we rebuilt the tree every now and then, it was just
> way too slow (it really bogs down your system).

I was thinking mostly of the unsorted case (or server-sorted) where you
could populate the tree beforehand, and just replace the _text_ in the
elements as the headers arrive, and no actual update to the structure of
the tree is necessary. Is that case just as bad? 

Presumably the etable could be fixed if dynamic updates were actually
_used_ -- given time/motivation/etc. of course. 

>  - partially its the abstraction layers used, although there is actually
> no technical reason why the current abstraction can't be used in a
> different way at a lower level and implement this behaviour.

>  - there's still the problem of how to display mail arriving with
> sorting on, which can make it next to unusable to use the display while
> new things get added.

TBH, I'd start with "don't" -- if you're doing local sorting, then just
do a full download of the headers beforehand, as we do now.

> The main reason is the first, the code hasn't been fast enough, although
> etable in 1.4 is looking significantly faster so it may become
> possible.  We might even get enough bugs out of it to incrementally
> build the tree rather than having to discard and rebuild from scratch
> every time, which would make it easy.

Nothing is as slow as my 64K ISDN line. I've watched it chunter away to
itself for ten minutes at a time downloading headers I didn't want to
see, even after I've changed my mind and selected a different folder --
surely the etable can't be _that_ bad? :)

Seriously -- sometimes I fire up PINE to read my mail while I'm waiting
for it :)

> The second issue is more work to fix, although, not impossible.  We just
> need to throw out 'folder updated' events during long processing, at the
> low level imap engine.

Why so? If we're incrementally building the tree why does it matter if
the folder changes?

> Lastly, I hate the IMAP code probably more than you do, i have to help
> maintain it :)

:) Everyone hates IMAP code -- it's what it's _for_. I'm getting to
quite like Evolution though, so I suspect I'll try to help out too.

-- 
dwmw2