Hi all, we're back with a bunch of questions. This is an email conversation I had with Matthew regarding Evo/EDS internals. I got his permission to post the message here, so everyone can participate in clearing things up for us. Matthew already gave some very valuable hints and directions on how to proceed with our project. Any further input is greatly appreciated. ---------- Forwarded Message ---------- Subject: Re: [Evo and Kolab2 - a fresh attempt] Evo/EDS and The Big Picture Date: Monday 21 June 2010, 17:57:18 From: Christian Hilberg <hilberg kernelconcepts de> To: Matthew Barnes <mbarnes redhat com> Hi Matthew, first of all, thank you very much for taking the time to write a much-detailed answer to my questions. Will it be okay to forward this email to evolution- hackers or should your answer be made available to the public in any other way (if at all you're okay with that)? I think it could help others, too. If it is okay with you to continue this on evolution-hackers, just slip me a short note and I will forward the message to the list, which is also why I did not remove any of your lines other than blanks :-). On Monday 21 June 2010, at 15:26:37 Matthew Barnes wrote: > On Mon, 2010-06-21 at 12:47 +0200, Christian Hilberg wrote: > > Since my initial postings on the mailinglist regarding our project on > > creating a Kolab2 plugin for Evolution, my colleague and I have been busy > > piecing together how Evo and EDS are working internally. We'll most > > likely settle on 2.30. So far, we have made some progress, but the big > > picture is still missing. We have been grovelling through Evo/EDS > > sources and read as much of the documentation we found searching the web > > as possible, but we still cannot seem to get a true grasp on how things > > play together, especially the data flow between Evo and EDS and the > > who-calls-whom-when-and-for-what. We're both new to Evo/EDS development, > > and we've already taken too much time figuring out how things work (i.e. > > we're behind schedule... =). > > Not sure how much you've figured out yet so let me just give you a very > high-level overview to at least get you pointed in the right direction. > > The Evolution-Data-Server package provides D-Bus services for contact > and calendaring data, along with client-side and server-side libraries > for each. We've read about D-Bus in several postings regarding the redesign of Evo/EDS (mainly in respect to replacing Bonobo), but we were not clear about D-Bus being in use for Evo-EDS-comm already. > The package also embeds a mail library called Camel. Camel > lives in the Evolution-Data-Server package only for historical reasons > and really deserves to be split off as a stand-alone library. It helps > to think of Camel as though it were a stand-alone library, unrelated to > the rest of Evolution-Data-Server. Okay, this is what we assumed reading through the mailing list archives and planning pages. that the Camel lib will be split off completely to be standalone sooner or later. > Complex as Evolution is, it's really just a graphical front-end for the > contact and calendar D-Bus services provided by Evolution-Data-Server, > as well as a graphical mail front-end for Camel. All rolled into one. The D-Bus services are a valuable hint, indeed. > The client-side libraries provided by Evolution-Data-Server are called > libebook for contact (i.e. address book) data and libecal for calendar > data. These are the libraries that front-ends like Evolution uses to > talk to the D-Bus services. We got the sources for Evo and EDS, both version 2.30.1, and ran Doxygen on them (just for the nice clickable images it generates using dot ;-). The include graphs and included-by graphs generated by Doxygen are helpful, but some things you can easily miss when you are not clear what to look for. Now, after your explanation, I checked the generated docs for addressbook/libebook/e-book.c (EDS source) and this file's #include <dbus/...> which just escaped me when I was digging through the docs since I did not expect it there. Which is why your answer is much helpful. > The server-side libraries are called libedata-book and libedata-cal for > contact and calendar data, respectively. These libraries are only used > by the two programs providing the D-Bus services: e-addressbook-factory > and e-calendar-factory. Each server-side library defines a subclassable > interface for storing and retrieving remote contact and calendar data: > EBookBackend and ECalBackend. That much we found already. It is good to know that we are not entirely going wrong. :-) > Your job for contact and calendar data is to subclass EBookBackend and > ECalBackend and implement their methods to interact with Kolab2. > Camel provides a similar subclassable API called CamelProvider. The > main difference between a CamelProvider versus an E-D-S backend is the > CamelProvider runs directly in the front-end process rather then behind > a D-Bus service (although there are long-term plans afoot to provide a > D-Bus service for mail). Now starts the fun. Kolab2 stores all calendar data in annotated IMAP folders holding email messages with XML attachments (at least one email / attachment per calendar entry). This means we will have to facilitate Camel for accessing calendar data. Hope there is no reason why this should not work (as you pointed out that the Camel lib is more or less standalone anyway). Does the "front-end process" refer to Evolution, i.e. is a Camel.Provider running inside Evolution instead of EDS? I was hoping that all non-UI-related stuff could be run inside EDS, so we would facilitate Evo only for account setup and displaying purposes. Now, if Camel.Provider would run inside Evo, this would complicate matters. Or maybe, we could get along without a Camel.Provider for IMAP access? Reading through http://www.go-evolution.org/Camel.Provider now, under "Initialisation", I find it states "By convention only, within Evolution, these will always be called from the 'main' thread, i.e. the Gtk main loop thread.", which now gives me the hint that the Camel.Provider runs inside Evolution. The phrasing would also allow for an interpretation that Camel.Provider lives within Evo *or* EDS. I would not have guessed that (my thinking was that, since the Camel sources reside within EDS, it will run there). Also, the website mentioned above does not give a hint on that matter other than the sentence I've quoted. For now I'll assume that the Camel.Proiver runs inside Evo and has to be used for IMAP access through Camel - is that correct? If in some future there will be some (funded) time to work on docs, it would really be helpful for the larval Evo/EDS hacker if the docs would specify the context(s) the documented subject resides in (e.g. that the Camel.Provider runs from within Evolution, not EDS). > CamelProviders come in two flavors: stores and transports. Stores are > for storing and providing access to email, such as IMAP. Transports are > for sending email from one host to another, such as SMTP. > For most proprietary groupware servers you would be implementing both a > custom store and transport, but if Kolab2 is entirely IMAP-based I'm not > exactly sure how that will work. In any case, your job for mail data is > to subclass CamelProvider and perhaps also CamelStore and CamelTransport > and implement their methods to interact with Kolab2. Calendar data and (shared) emails are retrieved using IMAP. Actions like setting up a meeting is done by sending an email containing an XML data attachment describing the meeting calendar entry to the Kolab2 server via SMTP. This email is then processed by the Kolab2 server and e.g. invitation emails are generated and sent to the invitees (to be precise: the Client do an IMAP poll to check for new messages). The invitees do accept or decline an invitation by replying to the invitation message accordingly. The task of setting up the appropriate XML message is handled by the client software (e.g. Kontact, or soon-to-be Evolution). This means, at least part of the "calendar backend" will reside in Evolution. We will have a helper lib which will convert Kolab2 XML format to Evolution data structures. This lib was planned to live within the EDS backend code. Now that I know that the Camel.Provider runs inside Evo, I'm not fully sure whether putting the conversion lib inside our EDS plugin would be the Right Thing(tm) to do. > The only thing you'll be writing for Evolution is an EPlugin to add > Kolab2 as an account type choice and maybe provide some custom options > for it. This is what we planned, indeed. But see above. > These plugins are usually pretty simple, and there's plenty of > examples to study in Evolution, Evolution-MAPI and Evolution-Exchange. Okay, we'll read on then. > > Maybe we have missed the right place to search for documentation on how > > to > > write a fully-fledged Evo/EDS-Plugin. We found > > http://projects.gnome.org/evolution/developer-doc/eplugin/ , but that > > covers the EPlugin side only and is, if I may say so, a little outdated. > > A similar document for the EDS side is missing, and so is one that > > covers Evo/EDS interplay. The source itself is commented very sparsely > > only (which means there are no misleading or outdated comments ;-) > > please don't get me wrong). It took quite a while to find out even which > > datatypes are responsible for holding contact data and calendar data (a > > work not yet finished). Aside from the pure API itself, which we can > > deduce from existing projects, the API's semantics are a hard part to > > find out. Much stuff from go-evolution.org is helpful, but (as far as > > we're concerned as n00b Evo/EDS hackers) not really sufficient. Mostly, > > when we search for information on > > http://live.gnome.org/Evolution, it's either identical to the > > go-evolution site or not yet existent. > > Documentation for our architecture is inadequate, and what little there > is was mostly written years ago. These days it takes all of our limited > manpower just to keep the code working, and little time is given to > documentation (and almost none to automated testing). Sadly, that state > of affairs is common for underfunded Open Source projects. Sadly, yes. > That said, the Evolution-Data-Server APIs (including Camel) are fairly > documented and browsable here: http://library.gnome.org/devel/references > Search the page for "evolution" and "camel". These references we found, and they also prove your point about underfunded OSS projects. > The EPlugin documentation you found is woefully out-of-date and needs a > complete rewrite, which is on my to-do list. You're better off mimicing > existing examples. This is what we will do. Skimming through the EPlugin documentation, it was showing it's age by referring to Bonobo every now and then. We'll check with evolution-mapi and friends for that matter. > http://live.gnome.org/Evolution is the new home for our wiki, replacing > go-evolution.org. All of the still-relevant content from go-evolution > has been migrated over, but it's not all linked to from the front page. > You might have to search. Pages names on live.gnome.org should be > similar to those on go-evolution.org. Okay, thanks. > There's a lot of code to digest, and after four years I'm still trying > to get my head around it all myself. Mimicing other backends is the > best way to bootstrap your project, and please don't hesitate to post > technical questions to evolution-hackers gnome org I work primarily on > the Evolution front-end and haven't acquired much experience with the > backend APIs, but I'll be sure to harass the other developers to be > responsive to your questions if I can't answer them myself. Please be sure not to harrass them too hard as it would be sad to have the project suffer not only from underfunding but also from an exodus of developers. :-) > Hope this helps, > Matthew Barnes Thank you again for your insights and the time you spent writing them up, they're invaluable. If I get your permission to forward this whole message to evolution-hackers, I will do, and discussion can continue there. If you want any paragraph to be deleted prior to forwarding, just mention it. ------------------------------------------------------------- Greetings, Christian -- kernel concepts GbR Tel: +49-271-771091-14 Sieghuetter Hauptweg 48 Fax: +49-271-771091-19 D-57072 Siegen http://www.kernelconcepts.de/
Attachment:
signature.asc
Description: This is a digitally signed message part.