Re: [Evolution-hackers] Introduction and Questions

[Jeffrey Stedfast <fejj novell com>]

On Mon, 2007-06-11 at 09:46 -0700, Ross Boylan wrote:
> I'm preserving the exchange for context; my responses are sprinkled
> below.
> On Sun, 2007-06-10 at 23:47 -0400, Jeffrey Stedfast wrote:
> > On Sun, 2007-06-10 at 15:55 -0700, Ross Boylan wrote:
> > > On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:
> > > > > 
> > > > > Second question: even if it creates a folder, does it need to stick
> > > > > around for the folder creation to finish?  I think I remember seeing
> > > > > that camel was single-threaded
> > > > 
> > > > not true...
> > 
> > I should have been clearer here:
> > 
> > yes, it needs to wait for the folder object instantiation to finish (the
> > function returns a pointer to the instantiated object, afterall).
> And your proposal with "open" on that object is designed to decouple
> creation of the object from creation of the summary, right?

correct, the ::open() would load the summary from disk and (if the store
is network based), sync state with the server (find out about messages
which were deleted since last session, flag changes, new messages, etc)

>   When I said
> "folder creation" I was thinking of the whole get headers and make index
> process.

ok

> > 
> > not true that camel is single-threaded - it has a few of its own threads
> > going - most notably is the thread for filtering incoming mail for
> > providers like imap and maildir (they share the same thread).
> > 
> > > > 
> > > > > , relying on the client app to do
> > > > > threading.  Would there be a way to multi-thread this somewhere
> > > > (either
> > > > > in camel or in the app)?  Obviously doing so would complicate
> > > > things,
> > > > > because at some point one might need to block (e.g., if I move a
> > > > message
> > > > > from folder A to B and then switch the UI to look at B).
> > > > 
> > > > okay, I think you need to familiarize yourself with Camel's API before
> > > > we discuss anything further :)
> > > > 
> > > http://www.go-evolution.org/Mail_Threading begins
> > > "The Camel API is blocking and synchronous"
> > > although it then goes on to qualify this:
> > > "Custom Threads
> > > Two tasks inside Camel use threads internally in order to implement a
> > > cancellable api ontop of a non-cancellable one."
> > > 
> > > Further
> > > "Mail-OPS
> > > mail-ops.h contains asynchronous versions of varous Camel api's"
> > > It sounds as if mail-ops is outside of Camel, however.
> > 
> > it is external to camel, yes.
> > 
> > > 
> > > So it sounds as if Camel could (in principle) respond to a move request
> > > by issuing the appropriate IMAP command and then, starting a thread to
> > > do the other activities (indexing the target folder and deleting the the
> > > message from the  source folder) and return.
> > 
> > it could, yes, but it'd need a way to report an error unless it waited
> > for the operations to finish before returning. For moving mail, you
> > typically want to know that it succeeded :)
> > 
> > all of the current camel APIs are designed such that the caller expects
> > that when the function returns, the operation has either succeeded or
> > failed (where the failure would be set on the CamelException argument).
> > 
> > >   It would then block on
> > > operations that attempted to access the target folder until the other
> > > operations completed.
> > 
> > yes, this is true... well, the way folders are implemented at this time
> > anyway...
> > 
> > > 
> > > I think this could be called a syncronous API, though perhaps that's a
> > > stretch.
> > 
> > it is indeed a synchronous API :)
> Syncronous, but it fails the "you know if you've succeeded when the
> function returns" test.

most of the camel APIs don't fail that test

> > 
> > > 
> > > On the other hand, http://www.go-evolution.org/Camel.Operation does not
> > > sound like a bocking syncronous API at all, so maybe the statement
> > > quoted at the top is just obsolete?
> > 
> > Camel code is written under the assumption that it is likely it is being
> > used in a thread, so it has CamelOperation as a means to:
> > 
> > 1. report the progress
> > 2. cancel operations (typically i/o)
> > 
> > what happens is Evolution has a thread-pool with a CamelOperation object
> > for each of the threads the mailer creates so that it can cancel
> > operations and/or get progress updates.
> This part of the camel API is not syncronous, right?

in so much as doing camel_operation_cancel() returning doesn't mean that
the other thread has stopped, correct. Basically it just means that
you've sent a command along a pipe to the thread to cancel at its
earliest convenience.

> > 
> > > 
> > > So, first of all I'm confused about the nature of Camel's API and
> > > operation as far as threading and syncronicity.  
> > > 
> > > Second, I don't have a sense of whether its features are historical
> > > accidents (camel was implemented in a simple way and evo then used it as
> > > it was)
> > 
> > evolution mailer was originally not multi-threaded if my recollection
> > serves correctly, although around the time of the first 0.0 release
> > threading was added (about 7 years ago).
> > 
> > camel and evolution were developed together, so their "designs" evolved
> > together.
> Why was camel separated out at all?

good model/view split :) you typically want to decouple low-level code
from user-interface elements

> 
> Side note: when I first was trying to figure out what was going on, I
> looked in the evo sources and discovered the code wasn't there, but was
> in the evo-data-server.  This was confusing because, at least on debian,
> the latter was described as providing calendar and address book
> functions.  I think the Debian split mirrors the what I see in your
> repository.  My current guess is that some of the code in
> evolution-data-server (the directory) is really for evolution, and not
> for the evo-data-server application.

evolution-data-server is basically the backends for address book and
calendar which runs as a daemon process for applications to query.

camel was shoved in there as well, even though it doesn't really work
like the others, because camel shared some of the convenience functions
with the addressbook/calendar parts and rather than duplicate that code,
just threw it in with e-d-s.

> > 
> > >  or the result of some deliberate design decisions.  Blocking
> > > syncronous operations are simpler to implement,
> > 
> > yep
> > 
> > >  to use,
> > 
> > yep
> > 
> > >  to debug,
> > 
> > yep, altho with threads thrown in and gdb multi-thread support sucking,
> > this was still a nightmare...
> > 
> > >  and to
> > > understand, so they clearly have some advantages.   But it seems that
> > > the entire application (evolution) does not have that character, so the
> > > benefits of that simplicity end up lost anyway.
> > 
> > I'm not sure I understand what you mean here...
> I was saying that "make camel syncronous; it's easier" would be a
> stronger argument if the entire application were syncronous.  Since it's
> not, making camel syncronous really only redistributes complexity rather
> than eliminates it. 

sure, but it was decided that it'd be far easier doing things this way
than making camel asynchronous. Was it the correct decision? *shrug*
We've certainly had our fair share of pain and suffering, but I can't
say with certainty that it would have turned out less painful had we
gone the other route.

> > 
> > I guess what you are saying is that evolution code isn't easy to
> > understand. 
> That's true, but as you say, it's true of most everything.  I wasn't
> making a criticism.

nod, I didn't take it that way :)

>   Given that I came in wondering about performance
> and responsiveness, I view the complexity of threads as a necessary cost
> to achieving a good user experience.
> 
> Speaking of complexity, evolution/camel (and maybe Gnome?) do a kind of
> objects with C.  Why not just use C++?  Was the C++ toolchain too
> unreliable at the start, or was there some other reason?

for a lot of reasons, I suppose... in no particular order:

1. symbol munging was always changing in the gcc toolchain
2. c++ is a lot higher barrier to entry (it's a lot harder to
understand, especially if developers go nuts with operator overloading,
templates, etc - I myself give up whenever I try to debug most c++ apps)
3. c++ gains you almost nothing... except 'class' and inheritance, but
that's trivially done in c anyway...
4. language bindings
5. it's the language the developers were most familiar with (for both
the GNOME and Evolution projects)

(you'll note that projects like Mozilla, even tho they use c++, use only
a very minimal subset which basically amounts to 'class')


certainly writing a multi-million-line project in c was not ideal, but
I'm not sure c++ would have made it much better :)

(it would have eliminated some of the boiler-plate code needed to
subclass in c, that's about it)

> 
> > I think that's true for all large/complex projects. I think
> > that it also feels that way when you are new to the project because it
> > is so large and overwhelming - I mean, I know that's how I'd feel if I
> > were just getting into Evolution sources...
> > 
> > I think that it probably doesn't help that things aren't super well
> > documented either (kinda hard to do when things change so much so
> > frequently). From what I can tell, most software (proprietary included)
> > suffers greatly from this problem :(
> > 
> > I think it's obvious that most of evolution's design is evolutionary
> > (hah, bad joke, eh?). Despite that, I think it actually turned out quite
> > good, but some APIs are certainly clumsy/awkward.
> > 
> > Anyways... it's always easier to criticize than it is to create.
> > 
> > Hopefully I've cleared things up.
> Getting clearer :)

cool :)

Jeff