[Evolution-hackers] Overview of camel-lite

[Philip Van Hoof <spam pvanhoof be>]

I'm going to make an overview of camel-lite so that if people read the
discussions here, that they know what it's all about.

Camel-lite is the fork of Camel that Tinymail is using internally right
now. Camel-lite's code is in my opinion not really in a good shape, it
more or less implements certain features that are necessary for
Tinymail.

Bad things:

 * Some of the features have been done in a hackish way. 

 * Mostly this had to do with non-blocked reads not being very well
   supported within the code and design of upstream camel

 * Removes support for a few for Evolution very important but for
   Tinymail unused features

 * It's a fork (effectively, yes)

Neutral:

 * I am trying to get some of the changes in upstream Camel. Although
   I'm selective because I don't want to bore the Evolution team with
   features that they'll most likely not consider for inclusion anyway.

   My point of view is that if they show interested, I'll bump the
   priority of trying to clean it up and getting it upstream

Support:

 * Push E-mail through IMAP IDLE, IMAP commands shutdown and restart the
   IDLE state of the connection. A non-blocking read handles EXISTS,
   FETCH and EXPUNGE events (you see messages being added almost
   instantly when they either get added or received)

 * Better detecting of EXPUNGE through / partly caused by IMAP IDLE (you
   see your messages getting removed almost instantly if you remove
   messages from the folder from another E-mail client)

 * Better detecting and handling of FETCH through / party caused by IMAP
   IDLE (flag changes are effectively working instantly with for example
   Cyrus: you see your view changing almost instantly if you work from
   another E-mail client with your folder)

 * Better progress indicating. Camel-lite doesn't use percentages but "n
   of nth" values. For example bytes of a message or items of a summary
   download

 * Support for BINARY fetching messages, which is more bandwidth
   efficient in some situations 

 * Support for CONDSTORE which is far more bandwidth efficient when
   synchronising folders

 * Support for receiving a message while summary is being downloaded.
   Camel-lite creates for this single-use-case a new socket (this is a
   very new feature that is being bug-fixed these weeks)

 * Support for displaying the recently received headers early while
   folder summary is being downloaded (very neat, and extremely useful
   with the parallel receiving of a message being functional now)

 * Stores already-received summary information very early, recovers from
   already stored information (saves bandwidth in case of a connection
   failure)

 * Makes the selection (the amount of columns or fields) of the summary
   download a lot smaller

 * Doesn't store a message temporarily in memory while retrieving it:
   immediately streams from the socket to the file stream in the cache.
   This is obviously an important memory-usage fix for large messages on
   mobile devices.

 * Implements summary support for POP3, makes the POP3 provider of Camel
   a full and real CamelStore that can be used just like the IMAP one.
   This, by the way, uses TOP for retrieving and generating the summary

 * Implements partial message retrieval (only retrieving the readable
   body mime part of a message) for both IMAP and POP3 (with POP3 the
   connection is forcefully cut, which might not work well with all POP3
   servers. Although this is regretfully one of the few only solutions
   for this type of feature with POP3)

 * A lot of bug fixes and locking improvements (also a lot of new
   situations to support and lock correctly, so it can also be seen as a
   non-improvement depending on your point of view -- stability vs.
   features --)

 * A lot of small memory-usage improvements (A lot of g_new's are now
   GSlice's)

 * Full replacement (> 95% of the allocations) of EMemChunk with GSlice

 * The CamelFolderSummary loaded in mmap, in stead of fread'ed into
   malloc'ed blocks of memory (this is of course a really big memory
   improvement in terms of allocated heap -- the kernel is really smart
   about mmap, and it's quite fast too .. indeed (probably faster than
   fread for loading, and ~ equally fast on a device with few memory
   resources during usage) --)

 * "All" compilation warnings fixed (we usually compile with -Wall and
   -Werror). Fixing this caused four major bug fixes in initialisation
    of variables in Camel.

Planned:

 * Improved CamelFolderSummary implementation (blocks of mmap-ed files
   and the flags in a different file)

 * QRESYNC support in parallel with CONDSTORE (Only Isode MBox is soon
   going to support QRESYNC, but it's cool so .. why not)

 * Fetching individual mime parts and letting CamelMimePart fetch
   requested mime parts (in stead of always fetching full messages to
   the cache)

 * NOTIFY support (read IMAP/Lemonade drafts)

 * CONTEXT support (read IMAP/Lemonade drafts)

 * Support for SEARCH over multiple folders (IMAP drafts, future)

 * Support for server-side SORT


Not planned:

 * Pipelining (maybe some day with or in the IMAP4, or with or in
   libspruce)



-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog