Re: [Evolution-hackers] Introduction and Questions

["Sankar P" <psankar novell com>]

On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
> On Thu, 2007-06-07 at 08:25 +0200, Gilles Dartiguelongue wrote:
> > Le jeudi 07 juin 2007 à 01:53 +0300, Philip Van Hoof a écrit :
> > > Without immediately writing to disk work, the download by itself will
> > > consume around 120 MB of RAM, and will most likely fail due to network
> > > timeouts and other such problems (it'll take a while, since Evolution
> > > fetches a ridiculous large amount of headers for each message for
> > > building its summary).
> > > 
> > isn't the imap-features plugin's goal to reduce/customize the amount of
> > downloaded headers ? Or is it that it's still not enough ?
> 
> It improves the situation by setting your url-string to have the
> "basic_headers" option. In the imap code of Camel, it will then ask for
> less headers (but still way too much).

May be way too much for a mobile client; but not for a desktop email
client. Sure you dont want your desktop mail client to have threading or
show mail-size or have such basic things ?

> 
> A major improvement it certainly isn't.

Try comparing the performance of fetching all headers and basic_headers
only. Some crude data which I collected a year back is at
http://psankar.blogspot.com/2006/05/imap-performance.html

> 
> The best way is to ask for the ENVELOPE and the remaining info using the
> normal BODY.PEEK method. It should be possible to specify per folder
> which of the headers are to be fetched, to make it really efficient.

Do you really think any user on earth will really be interested in
configuring what headers to fetch on a folder-basis ? 

> 
> The imap4 implementation seems to have an ENVELOPE parser, so I guess
> either it does it already or it will use ENVELOPE in future.
> 
> For a mobile device, that works over GPRS, you for example are usually
> not interested (not at all) in the mailinglist specific headers.

None of the mailing list headers will be fetched if you have chosen the
basic_headers option.

> 
> It would also be possible to do it in multiple passes. For example get a
> modest list of headers, and after that get more and more.
> 
> In any case, none of the current Evolution code implements consuming the
> CONDSTORE capabilities of some modern IMAP servers (like MBox and
> Cyrus).
> 
> CONDSTORE is really going to make an enormous difference in bandwidth
> consumption with large folders. That's because you only need to fetch
> the flags of the changed messages to synchronise flags.
> 
> Note that Tinymail's camel-lite implements all the needed solutions to
> this bandwidth consumption problem. And that its code is, although a lot
> of work because the Evolution maintainers didn't seem interested at the
> time it was happening, definitely portable to upstream.
> 
> Its implementation include solutions for the headers, it immediately
> saves the headers to disk and unlike Evolution's Camel it can resume
> partial summary downloads, it wont peek memory allocation during
> downloading, it implements Push E-mail using IMAP IDLE and it implements
> CONDSTORE.
> 
> Although I'm definitely not satisfied with any of either Camel nor
> camel-lite's IMAP code. The thing is that I'd much prefer a client-side
> implementation that does proper pipelining. For example Infotrope,
> Telomer and Polymer does this on an experimental basis (Infotrope is the
> library).
> 
> ps. In my opinion is also the imap4 project getting the majority of its
> design wrong. Although it looks better than the "imap" one, it's not the
> best way to use an IMAP server. If a project is going to write an IMAP
> client implementation 'today': they better do it right. A lot is
> changing in IMAP today (Lemonade, MORG, etc). It's important to get it
> right this time (the vast majority of E-mail clients totally suck at how
> they use the IMAP server, including Evolution indeed).

Such a sweeping generalization !?

As you mentioned, IMAP (like any other technology) grows at a very
high-speed. What you consider as the "right client implementation" today
may become obsolete tomorrow. When the initial IMAP provider was written
there may not be a standard spec. for the PUSH/IDLE etc.

No application can have an intelligent design that will remain forever
satisfying all new needs/changes. Application designs should just
evolve, adapting to the new changes.


IIRC, you had a branch on Evolution's Camel to work on whatever changes
you wanted to make, so that you don't have to wait for the delay due to
review/maintainer.  Would love to see some of the improvements that you
have done ported there. So that it becomes easy for the maintainer to
approve them and bring onto trunk.

> 
> 
-- 
Sankar

 Novell, Inc. 
Software for the Open Enterprise*
http://www.novell.com