[Evolution-hackers] Camel Manifesto

[Matthew Barnes <mbarnes redhat com>]

With work on Bonobo removal wrapping up, I've finally started taking a
closer look at Camel (Evolution's mail storage and networking library)
and laying out plans for where I'd like it to go over the short and long
term, with the ultimate goal of splitting it off as a useful standalone
GNOME library (but we're a long way from that).

As usual I'm taking a bottom-up approach, starting with basic cleanup
chores (both code and development policies) and building up from there.
Here's some of my thoughts:


Backward Compatibility Policy
-----------------------------

A reverse dependency search in Debian and Fedora reveals the only
external projects currently linking to Camel are mail-notification,
evolution-jescs and Anjal (please correct me if I've missed any).

That tells me that until Camel moves out of its parents' basement and
matures into an real, honest-to-goodness library, fixing its interface
is more important than maintaining backward compatibility.  Deprecation
periods for obsolete API are not necessary, in my opinion.  The few
external projects linking to Camel will just have to keep up with the
changes every six months.

That's not a license to go hog wild though.  Some caveats:

1) The soname -must- be kept accurate.  If you break the API or ABI,
   increment the soname when you commit the break.  It doesn't matter if
   the break doesn't affect external projects, nor even if it's already
   been incremented since the last point release.  Bump it anyway. 
   "Always bump" is an easy policy to remember.  It makes our own daily
   development run smoother, and helps ensure a release doesn't slip out
   with an inaccurate soname.

   If you're not sure if your patch requires a soname increment, please
   ask in IRC or Bugzilla.  Patch reviewers should try to remember too.

2) If you must break the API, try to do so in a way that things will
   fail noisily at build time rather than mysteriously at run time.  For
   example, if you want to change the behavior of an existing function,
   it's better to rename the function or change its parameter list so
   that stale Evolution code will fail to build.

3) Camel started life as a general purpose mail library and I'd like to
   try to get back to that.  Camel has become too Evolution-centric in
   my view, with too many quick-fix hacks for Evolution bugs that would
   not be appropriate for a general purpose mail library.  I will clean
   these up as I find them, but try to keep that in mind when altering
   the API yourself.


Migrate to GObject
------------------

Camel's homegrown type system will be replaced with GObject so that
introspection and D-Bus + language bindings are possible.  CamelObject
will remain (for now), but as a subclass of GObject.  The redundant
parts of CamelObject will be removed.

I'm also following GLib and GTK+'s example of sealing up public instance
data in private sections and enforcing that only its top-level header
file be included outside of Camel (including the providers).  Unlike
GLib and GTK+, there will be no transition period.

This will give us a lot more freedom to reorganize the library and
refactor code without disturbing the ABI.  Debugging is also easier when
you can trap data accesses through "get" and "set" functions.

I've been chipping away at this as a side project for the past year (it
was a good mindless activity when I got burned out on Bonobo removal),
and I recently published my results to git.gnome.org as a branch named
"camel-gobject".  The CamelObject conversion is finished -- including
all the boilerplate changes in the subclasses -- but I haven't finished
sealing up the API.

The branch probably won't land until 2.31 at the earliest.  The backward
compatibility policies I described above would be in effect thereafter.


Kill CamelStream
----------------

This is a distant future goal and will have to happen gradually, but I
would like Camel to shift to a single-threaded design where all file and
network operations directly use or are derived from GIO's asynchronous
file and stream APIs.  SSL support is currently under development for
GIO, and that's the only missing piece I see at the platform layer.

I realize this is a drastic course correction and will require rewriting
all the providers and much of the mailer code in Evolution, but I firmly
believe that the overuse of threads carries much of the blame for
Evolution's chronic instability over the years and that reversing that
trend first and foremost requires minimizing our use of threads for I/O
and relying more heavily on GLib's main loop.


Comments and constructive criticism encouraged.

Matthew Barnes