Re: [Evolution-hackers] A Camel API to get the filename of the cache, also a proposal to have one format to rule them all

[Philip Van Hoof <spam pvanhoof be>]

On Mon, 2009-01-05 at 09:41 -0500, Jeffrey Stedfast wrote:
> Philip Van Hoof wrote:
> > On Mon, 2009-01-05 at 08:25 -0500, Jeffrey Stedfast wrote:
     
> >
> > Maildir doesn't store individual MIME parts separately. So Mailbox is
> > equally hard to handle for metadata engines as MBox is. Only difference
> > with MBox is that we need to seek() to some location.
> >
> > So Maildir doesn't make it possible for us to let app developers
> > implement indexing plugins easily, like a typical exif extractor.
> >   
> 
> I guess, but they could just link with gmime or camel :p

Which is what Tracker is doing at this moment. But for various reasons
we still end up copying the E-mail's decoded attachments to /tmp, then
scan them with the indexer's plugins, and then unlink() the files.

Suffice to say that this ain't ideal when scanning 10.000 E-mails that
way. Much more efficient for us would be to simply enter evo's caches
and read the MIME parts as normal already decoded files.

I also think such a format would improve some of Evolution's own
features:

o. For example a making a thumbnail of an image could use the platform's
   infrastructure, and see it being cached using the thumbnail-spec.

   Less code

o. Another feature is the "Save as" feature for attachments. Instead of
   having to open a GFile and using CamelStream converted to a
   GOutputStream and decode-streaming it to that stream to save the
   attachment on the filesystem, you just copy the file.

   Less code

o. Inline image viewers: Instead of having to plug the decoded memory of
   the attachment into a blob of memory, you just use any image viewer. 

   Less code

o. Inline attached images for text/html MIME part viewers: right now
   migrating GtkHTML to WebKit or GtkMozEmbed is hard because GtkHTML
   had implemented some special thing that allows it to get itself a
   blob of memory fed as pixmap buffer for images whom src attribute
   start with "cid".

   Less code


I'm not even sure if WebKit and GtkMozEmbed support rendering blobs of
memory. Although I have been asking the developers of the respective
components at nearly each conference I meet them about this. They all
promised to at least offer some sort of infrastructure for this.

Lot's of promises ;)

After thinking about it very hard, and quite a lot, I didn't find any
good reason to store attachments in Base64 encoding. I only found
reasons why you would want to store it decoded: Less code, same features

The only exception why storing in Base64 encoding could be the feature:
"View the source of this E-mail". You can perform the Base64 encode as
the E-mail becomes visible in the E-mail source viewer, it's not a good
reason (let's say this introduces 5 lines of camel_stream_* code).

You could say: because we want to use a "standard" for our storage:

 - Mailbox can't work on Windows because the author of the spec refuses
   to change the character ':' into '!' for the filenames. Which renders
   his entire specification completely useless. Windows is not
   irrelevant, it's being used a lot. Ignoring it is like carving the
   word stupid on your head with a knife.

   But fine, let him. We are free to ignore his spec, right?

   Maildir also doesn't specify storing MIME parts as separate files.

 - MBox is just broken. You can't put 3Gigs of data in one file, require
   a rewrite of that file each time you want to remove 1kb of data out
   of it and have no index on it (this, at least, is something Maildir
   got right by letting the kernel's FS take care of that: atomic
   renames and DIR is quite good as an index).

   An MBox file is a ticking timebomb waiting to get corrupted.

   MBox also doesn't specify storing MIME parts as separate files.

 - What other formats do we have? Is there one "so called" "standard"
   format that stores MIME parts as individual "decoded" files?

   Because if not then just like the Maildir-guy I'll quickly make a
   website and give it a name. And then let's all start calling it a
   "standard". Problem solved? It's not that Maildir is really that much
   more than that. A website that describes a broken way of storing
   E-mails.

   Well, ok, a few IMAP server guys decided to use that specification to
   shut up people who say that IMAP servers that store in a binary
   format are not compatible with their freedom religions. Of course
   that's an ill-educated point of view, but who cares. Freedom! *sigh*

-- 
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://pvanhoof.be/blog
http://codeminded.be