Refactoring

[Philip Van Hoof <spam pvanhoof be>]

After Tinymail's 1.0, I would like to focus for a certain amount of time
on cleaning up and refactoring the code.

My plan is to release a 1.1 that maintains ABI and hopefully API
stability too (hopefully both). My plan for this release is to cleanup
the code of the sections that I'll list here.

Meanwhile new features will be developed in a 2.0 branch. I'm for
example working on a bodystructure parser that'll open the doors for a
lot of fancy new stuff. I'm also planning to rewrite the summary store
to cope more easily with certain difficulties that it has right now
(that's another lengthy post by itself to explain, I'm going to spare
you guys).

Obviously will 1.1 patches always be merged to 2.0

Some things that have made the code ugly:

 * The TnyCamelQueue
 * The TnyIdleStopper
 * Calling the mainloop, this is related to both the queue and the idle
   stopper
 * libtinymail-maemo and its device implementations 
 * Camel IMAP's IDLE implementation
 * The entire Camel Maildir implementation
 * Camel IMAP's Fetching messages code
 * Camel IMAP's LIST code
 * We're doing quite bad at Unit Testing

The TnyCamelQueue
-----------------

What is ugly about it?

o. There is a lot of code duplication between each and every async
   function. Each of them need to call their callback in case of a
   cancellation, in case of a failure and in case of a success.

   In case of a failure each of them set err, in case of a cancel each
   of them set a cancelled variable.

   All that administration is duplicated for each async function.
   There's a lot of sharable code here.

o. Each of the async functions also have their own "Info" struct to put
   information in. It would be much nicer if we'd implement some sort of
   stack where we push those things in, and pop them off in the
   functions.

The TnyIdleStopper
------------------

What is ugly about it?

o. It feels and smells like a hack to me. But that's of course just a
   subjective opinion like any other.

o. I'm sure we can come up with something more standard in glib? 

	o. Perhaps we should add GDestroyNotify callbacks everywhere in
	   stead?


Calling the mainloop
--------------------

o. At some point we'll have to abstract the usage of the mainloop to
   support platforms that don't have GMainLoop. Right now this usage is
   much more limited and under control as most people think. In fact can
   you, by adapting the three "execute_callback" functions, control 99%
   of Tinymail's GMainLoop usage.

   Sounds good? Yes. But we nevertheless have three points in stead of
   one, and all three of them do exactly the same thing. We have no
   abstraction for a mainloop, so we can't allow an application dev. or
   libtinymailui implementer to provide one.

   I know a lot of glib fans will disagree with the necessity for this
   (hey Rob!), nonetheless is this actually needed indeed.

o. I'm planning to port Tinymail to WinCE, SymbianOS (P.I.P.S.) and
   Android soon. Afaik do none of those come with Gtk+ nor are their UI
   programs being ran by a GMainLoop.

   I'm also planning to start providing Qt models for QTreeView, and to
   start providing support for System.Windows.Forms as soon as I have
   a .NET language binding.

   Note. This doesn't mean that I'll stop loving GNOME.


libtinymail-maemo
------------------

o. I recently split up the dummy device and the conic device into two
   source files. Sergio picked this up (I think?) and made some
   improvements to all this. There are some problems with integrating
   Conic with the mainloop, etc etc

o. At least one vendor (Ubuntu Mobile) expressed their desire to have
   implementations of libtinymail-maemo that don't use Conic.

o. We'll need implementations for the hacker edition of the 770 (unless
   this comes with Conic)


Camel IMAP's IDLE implementation
--------------------------------

o. The current implementation uses an ugly usleep() and a non-blocking
   read(). This can be enhanced with a select() or poll() loop.

o. I have a patch for this ready, but there are a few discrepancies with
   it that need to be fixed first (else things will stop working
   completely, so this stuff ain't committed)

o. Cope with NOTIFY, this is also being done by me in my spare time.
   Coping with folder creations, deletions and renames is being
   implemented.

Camel IMAP's Fetching messages code
-----------------------------------

o. Let's completely redo this code. It's a total CamelMess.

Camel IMAP's LIST code
----------------------

o. Let's completely redo this code. It's a total CamelMess.

o. LIST-EXTENDED will soon give us correct unread and total counts
   without having to ask for STATUS: that's awesome! Less roundtrips and
   fast, accurate folder items in the folder-model.

   With the current code, we might as well just want to rewrite the
   CamelMess and put the LIST-EXTENDED extension for subscribed and
   unsubscribed states of a folder in there too.

o. Same thing for NAMESPACE, btw.

We're doing quite bad at Unit Testing
-------------------------------------

o. We are not writing a lot of Unit tests

o. We are suffering from that, because we've seen a lot of bugs. Often
   the type of bugs that would have been caught if we'd tested more.

o. Most of the participating developers give bad debugging feed back,
   because their debugging tools are either broken or lacking on their
   target platform. With Unit Test, we would have extra information
   about the problem. Which would make identifying the problem a lot
   more easy.

   Having them would save us more time than making them

o. I know I promised an agile development methodology that would have
   forced all of you guys to deliver a new unit test with each patch.

   I didn't. I was too weak, you guys where too strong (and I didn't do
   it myself either, which weakened my case even more)

   Nonetheless .. let's try to recover from this mistake and create more
   unit tests.

   GLib will soon ship with a unit test library, and Check is doing fine
   too. So we have sufficient tools for this. So comon guys! Testing is
   important. It really is.



So .. who's with me on this? Igalians? Openismus? Nokia perhaps?



ps. 


I do think it's becoming more and more necessary to clean up things.
We've seen a long period of focus on bugfixing after an even longer
period of feature implementation.

Refactoring happened in between those large iterations and was usually
too short in time to achieve really good results (although the
TnyCamelQueue refactoring that me and Rob appeared to be a huge success
in terms of stability)


We need to clean up the mess soon. Drain the swamp.


-- 
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://pvanhoof.be/blog
http://codeminded.be