Re: [Evolution-hackers] Introduction and Questions

[Ross Boylan <ross biostat ucsf edu>]

I'm preserving the exchange for context; my responses are sprinkled
below.
On Sun, 2007-06-10 at 23:47 -0400, Jeffrey Stedfast wrote:
> On Sun, 2007-06-10 at 15:55 -0700, Ross Boylan wrote:
> > On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:
> > > > 
> > > > Second question: even if it creates a folder, does it need to stick
> > > > around for the folder creation to finish?  I think I remember seeing
> > > > that camel was single-threaded
> > > 
> > > not true...
> 
> I should have been clearer here:
> 
> yes, it needs to wait for the folder object instantiation to finish (the
> function returns a pointer to the instantiated object, afterall).
And your proposal with "open" on that object is designed to decouple
creation of the object from creation of the summary, right?  When I said
"folder creation" I was thinking of the whole get headers and make index
process.
> 
> not true that camel is single-threaded - it has a few of its own threads
> going - most notably is the thread for filtering incoming mail for
> providers like imap and maildir (they share the same thread).
> 
> > > 
> > > > , relying on the client app to do
> > > > threading.  Would there be a way to multi-thread this somewhere
> > > (either
> > > > in camel or in the app)?  Obviously doing so would complicate
> > > things,
> > > > because at some point one might need to block (e.g., if I move a
> > > message
> > > > from folder A to B and then switch the UI to look at B).
> > > 
> > > okay, I think you need to familiarize yourself with Camel's API before
> > > we discuss anything further :)
> > > 
> > http://www.go-evolution.org/Mail_Threading begins
> > "The Camel API is blocking and synchronous"
> > although it then goes on to qualify this:
> > "Custom Threads
> > Two tasks inside Camel use threads internally in order to implement a
> > cancellable api ontop of a non-cancellable one."
> > 
> > Further
> > "Mail-OPS
> > mail-ops.h contains asynchronous versions of varous Camel api's"
> > It sounds as if mail-ops is outside of Camel, however.
> 
> it is external to camel, yes.
> 
> > 
> > So it sounds as if Camel could (in principle) respond to a move request
> > by issuing the appropriate IMAP command and then, starting a thread to
> > do the other activities (indexing the target folder and deleting the the
> > message from the  source folder) and return.
> 
> it could, yes, but it'd need a way to report an error unless it waited
> for the operations to finish before returning. For moving mail, you
> typically want to know that it succeeded :)
> 
> all of the current camel APIs are designed such that the caller expects
> that when the function returns, the operation has either succeeded or
> failed (where the failure would be set on the CamelException argument).
> 
> >   It would then block on
> > operations that attempted to access the target folder until the other
> > operations completed.
> 
> yes, this is true... well, the way folders are implemented at this time
> anyway...
> 
> > 
> > I think this could be called a syncronous API, though perhaps that's a
> > stretch.
> 
> it is indeed a synchronous API :)
Syncronous, but it fails the "you know if you've succeeded when the
function returns" test.
> 
> > 
> > On the other hand, http://www.go-evolution.org/Camel.Operation does not
> > sound like a bocking syncronous API at all, so maybe the statement
> > quoted at the top is just obsolete?
> 
> Camel code is written under the assumption that it is likely it is being
> used in a thread, so it has CamelOperation as a means to:
> 
> 1. report the progress
> 2. cancel operations (typically i/o)
> 
> what happens is Evolution has a thread-pool with a CamelOperation object
> for each of the threads the mailer creates so that it can cancel
> operations and/or get progress updates.
This part of the camel API is not syncronous, right?
> 
> > 
> > So, first of all I'm confused about the nature of Camel's API and
> > operation as far as threading and syncronicity.  
> > 
> > Second, I don't have a sense of whether its features are historical
> > accidents (camel was implemented in a simple way and evo then used it as
> > it was)
> 
> evolution mailer was originally not multi-threaded if my recollection
> serves correctly, although around the time of the first 0.0 release
> threading was added (about 7 years ago).
> 
> camel and evolution were developed together, so their "designs" evolved
> together.
Why was camel separated out at all?

Side note: when I first was trying to figure out what was going on, I
looked in the evo sources and discovered the code wasn't there, but was
in the evo-data-server.  This was confusing because, at least on debian,
the latter was described as providing calendar and address book
functions.  I think the Debian split mirrors the what I see in your
repository.  My current guess is that some of the code in
evolution-data-server (the directory) is really for evolution, and not
for the evo-data-server application.
> 
> >  or the result of some deliberate design decisions.  Blocking
> > syncronous operations are simpler to implement,
> 
> yep
> 
> >  to use,
> 
> yep
> 
> >  to debug,
> 
> yep, altho with threads thrown in and gdb multi-thread support sucking,
> this was still a nightmare...
> 
> >  and to
> > understand, so they clearly have some advantages.   But it seems that
> > the entire application (evolution) does not have that character, so the
> > benefits of that simplicity end up lost anyway.
> 
> I'm not sure I understand what you mean here...
I was saying that "make camel syncronous; it's easier" would be a
stronger argument if the entire application were syncronous.  Since it's
not, making camel syncronous really only redistributes complexity rather
than eliminates it. 
> 
> I guess what you are saying is that evolution code isn't easy to
> understand. 
That's true, but as you say, it's true of most everything.  I wasn't
making a criticism.  Given that I came in wondering about performance
and responsiveness, I view the complexity of threads as a necessary cost
to achieving a good user experience.

Speaking of complexity, evolution/camel (and maybe Gnome?) do a kind of
objects with C.  Why not just use C++?  Was the C++ toolchain too
unreliable at the start, or was there some other reason?

> I think that's true for all large/complex projects. I think
> that it also feels that way when you are new to the project because it
> is so large and overwhelming - I mean, I know that's how I'd feel if I
> were just getting into Evolution sources...
> 
> I think that it probably doesn't help that things aren't super well
> documented either (kinda hard to do when things change so much so
> frequently). From what I can tell, most software (proprietary included)
> suffers greatly from this problem :(
> 
> I think it's obvious that most of evolution's design is evolutionary
> (hah, bad joke, eh?). Despite that, I think it actually turned out quite
> good, but some APIs are certainly clumsy/awkward.
> 
> Anyways... it's always easier to criticize than it is to create.
> 
> Hopefully I've cleared things up.
Getting clearer :)
Ross