... and the pleasures of locking problems

[Philip Van Hoof <spam pvanhoof be>]

Tinymail's _async functions where a design error right from the start.
Therefore I drastically refactored them. I once also started
libtinymail-queues and the TnyQueue + libtinymail-asyncworker work to
emphasise that the _async functions are indeed a problem. I figured that
because a lot people are nonetheless going to use them, the big f.
design error in Tinymail needs a fix too.

so ...

I introduced an internal private type called TnyCamelQueue in
libtinymail-camel. Right now, in stead of simply launching threads in
the _async_default functions, I call _tny_camel_queue_launch.

The launch method of TnyCamelQueue is indeed the method to add an item
to the queue.

I haven't defined any contract for this method as it's a private
function. The contract for it can be considered to be:

"Will launch the function and pass data to it in future. Will
immediately return and wont block until the function is handled. The
function can happen in a thread therefore enforcing you to use locking
correctly in your implementations. Items in the queue will happen
serialised. This means that no items in the queue will be processed in
parallel." ... or something like that.

It does not specify when exactly in future, only that it will happen in
future. There's no priority setting, it's a simple FIFO.

All the callbacks of the existing _async functions are specified to be
called in the mainloop. What makes this indeed happen is (still)
implemented by the execute_callback function (which got recently
refactored by Sergio, in case you wonder about it) (differently put:
nothing got changed).

This might have broken some of the cancels and definitely has broken
support for parallel getting messages on IMAP. The last one is easily
solvable and fixable. The first one, in case true, is less easy to fix.

I noticed that the first one (broken cancels) doesn't always occur. I
noticed that the second one (broken parallel getting of messages over
IMAP) always does occur. I noticed that on POP parallel getting of
messages is still functional.

Since now _async operations happen serialised, there's a more clear and
clean view over the locking order. The amount of threads that can be
active is also drastically lowered to ... one for this queue.

I know I'm going to be stormed by panicking project managers and people
with lots of questions. This is better, more stable and worth
temporarily breaking the two functions (cancels and parallel getting
messages).

Although it wont fix all locking problems, it will or should fix a lot
of them or at least finding them and fixing it much more easy.

More work on also the account initialisation and connecting will happen
soon. This work attempts to get the alert_func, get_pass_func and
forget_pass_func to be ran in the GMainLoop of your application. This
could, at least in theory, also fix a lot of ui hang difficulties.


Last but not least: if people disagree that this is the right way to fix
the problems, they can ask me to create a branch somewhere that doesn't
redo the _async function's infrastructure.

We can also agree to disagree by having multiple implementations of
TnyCamelQueue. The first-original implementation simply calls the
g_thread_create API exactly like how the original code was, just wrapped
by the _tny_camel_queue_launch (makes absolutely no difference).

Note, though, that this affects the behaviour of Tinymail significantly
and that people who disagree will have to maintain the 'reverse of the
refactor' themselves.



-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog