[Evolution-hackers] IMAP summary synchronization improvements

[Philip Van Hoof <spam pvanhoof be>]

About

http://cvs.gnome.org/viewcvs/evolution-data-server/camel/providers/imap/camel-imap-folder.c?rev=1.363&view=markup

It looks like after 

seq = camel_folder_summary_count (folder->summary);

The "seq" is (often) 0 the first time an IMAP folder is used during the
session. After that, the summary is indeed loaded and seq will have the
correct value (the amount of summary items that are on-disk cached).

A consequence is that the first time an IMAP folder's summary is updated
(being online, that's right after Evolution is started up, for each
folder) the code will ask for-each UID rather than only the UIDs that
aren't yet in the summary cache.

A simple this thingy seems solve it:

i = camel_folder_summary_count (folder->summary);
if (i <= 0)
   camel_folder_summary_load (folder->summary);


FYI, I have reimplemented the "imap_update_summary" method here:

http://svn.tinymail.org/svn/tinymail/trunk/libtinymail-camel/camel-lite/camel/providers/imap/camel-imap-folder.c

This implementation uses a lot less bandwidth, consumes far less memory
and seems to be actually faster than the original one (although I do
dump the summary to disk each nth header).

It's faster because I send-and-receive a lot less with the IMAP service.

I fall-back on using the "UID FETCH %d:* (FLAGS)" in case it didn't
receive enough new headers yet (it calculates this). In normal
circumstances it will use "UID FETCH %d:%d (FLAGS)", immediately store
those UIDs in needheaders and in-one-loop create the CamelMessageInfo
instances and dump them to disk each nth header.

I removed all the headers that aren't being used. For example the first
"UID FETCH" doesn't need anything other than the UIDs.

In stead of three different loops that need to keep a copy of all the
data around during the whole transfer, I reimplemented this in one loop.

This also means that I can freeup the received data in-that-loop, rather
than out-of-the-loops. Keeping memory consumption at a minimum.

Together with the mmap patch, the dumping will mean that it will
flush-away newly received headers to the disk, that the mmap will reload
them and that memory consumption of those newly received headers will
therefore each nth received header be the same as the for-mmap-normal
situation (56% is mmapped and 44% is allocated).

The only API call that makes this incompatible with a normal Camel is
the camel_folder_summary_dump_mmap. If you diff camel-imap-folder.c of
Evolution's Camel with this camel-imap-folder.c of Tinymail's Camel, and
remove those API calls, you'll have a working Camel that doesn't use the
mmap stuff but that will receive and update summary faster from IMAP,
using less memory.



-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
work: vanhoof at x-tend dot be
blog: http://pvanhoof.be/blog