Re: [Evolution] Problem upgrading from Evolution 1.4.6 to 2.12.1

["R. Steven Rainwater" <srainwater ncc com>]

On Fri, 2007-11-09 at 16:46 -0600, R. Steven Rainwater wrote:
> I've been using Evolution v1.4.6 on a RedHat 9 box for years and finally
> decided it was time to move to something newer. I've installed Fedora 8,
> which includes Evolution v2.12.1. I copied over my /evolution directory
> hoping Evolution would notice it on start up and magically upgrade it.
> No such luck. It did see it but immediately generated this error...


I worked on this problem most of the weekend and have made some progress
that may help others in my situation.

I left out something in my original post that I realize now is quite
important. My Evolution 1.4.6 was configured to store its data in
maildir format. This is directly related to my upgrade problem because
support for user configurable local storage formats was dropped
somewhere along the way between 1.4.6 and 2.12.

For those who don't remember, Evolution 1.4.x allowed the user to select
the local storage format on a per-folder basis. By right clicking on any
folder you could select one of three storage formats: Maildir, mbox, or
mh. This per-folder control was incredibly useful and handy. Among other
things, it doubled as an easy way to convert from one format to another.
If you had an mbox file and needed Maildir, you just shoved it into an
Evolution folder, right clicked, and it was converted.

I used the feature to store all folders in Maildir format. This ability
was one of the main reasons I selected Evolution 1.4.x as my email
program of choice. I wish I had followed the development more closely so
I could have voiced support for this feature when the developers decided
to remove it.

If any of the Evolution developers read this list, please consider 
restoring the ability to select the local storage format.

Anyway, back to my upgrade story. In Evolution 2.12 maildir and mh are
no longer supported formats for the local data store, it only supports
mbox format. I'm not willing to give up Maildir format as I've found it
to have performance and data integrity advantages over mbox. So I've
spent the last couple of days looking for a work around. I've come up
with one that is tolerable, though it's an ugly hack.

While maildir support for local file storage was dropped from Evolution,
maildir support as an inbound "Server Type" was added. What this means
is you can select maildir as an alternative to POP3 or IMAP on the
inbound server type selector.

At first this seemed useless to me because my inbound server is POP3. If
I selected POP3, I couldn't have maildir support. If I picked maildir, I
couldn't retrieve my mail. Then I realized I might be able to create a
second account and kludge the two together somehow. So, Account 1
receives email from my POP3 server. Account 2 points to a Maildir area
on my hard disk and acts as an alternate local datastore. I then had to
create filters that move inbound mail from Account 1 to Account 2. 

The first problem with this arrangement is that you end up with two
inbox, junk, and trash folders, one in each account. The ones in Account
two are not needed but there's no way to delete them. This is a minor
annoyance, so I'm just ignoring it for now.

The second issue I ran into is that the Evolution-specific method of
nesting subfolders in a maildir area has changed between Evolution 1.4.6
and 2.12. (sadly it didn't change to the standard maildir++ format but
to yet another Evolution-specific method). I had a working example of a
1.4.x maildir layout but couldn't find any documentation on the
Evolution 2.12 maildir layout, so I had to grab the source code and poke
around to figure it out.

The Evolution 1.4.6 maildir format looked something like this (assuming
word wrap doesn't hose it up):

  MailDir/
    InBox/
      mbox/
        cur/
        new/
        tmp/
    FolderFoo/
      mbox/
        cur/
        new/
        tmp/
      subfolders/
        FolderBar/
          mbox/
            cur/
            new/
            tmp/
        FolderBaz/
          mbox/
            cur/
            new/
            tmp/

This layout prevented name collisions between the names of the required
maildir directories and any subfolders. Evolution 2.12 switched to this
seemingly less robust method that breaks if you name a subfolder cur,
new, or tmp:

  MailDir/
    cur/
    new/
    tmp/
    FolderFoo/
      cur/
      new/
      tmp/
      FolderBar/
        cur/
        new/
        tmp/
      FolderBaz/
        cur/
        new/
        tmp/

The mysterious cur, tmp, and new that show up in the maildir root seem
to correspond to the inbox folder. I'm not sure why it no longer gets an
explicitly named directory.

To convert your 1.4.x maildir datastore for use with Evolution 2.12,
you'll need to move around the folders a little. A bash or Perl script
can take care of this if you have hundreds of folders (as I did).
Remember to rename any folders named cur, new, or tmp to prevent
breakage. Also, note that 1.4.x dumped some index files into the maildir
directories. You can just delete those. Evolution 2.12 will re-index
them anyway.

Once I converted my maildir layout to the new format, Evolution could
see it without coughing up error messages. 

One additional problem cropped up at this point. A folder named
".#evolution" keeps appearing in the folder list. If I click on it, it's
empty. If I look at the actual contents of the directory in the maildir
area, it contains a file named "Junk.cmeta". I can delete the folder in
Evolution but it returns the next time the program starts. I presume
this is used to store some sort of internal information, in which case,
it probably shouldn't be visible to the user in their list of mail
folders. I filed a bug report on this:

 http://bugzilla.gnome.org/show_bug.cgi?id=495986

Otherwise, I've noticed three other minor annoyances in the move from
1.4.x to 2.12:

- No more "reply to list" option on the context menu. In 1.4.6, the
message context menu gave three reply options: "reply to sender", "reply
to list", and "reply to all". In 2.12 there are only two options: "reply
to sender" and "reply to all". I subscribe to many lists and used the
"reply to list" option frequently. Now to reply to a list message, I
have to go through a multi-step process: click the message to highlight
it, click "Message", then click "Reply to List". Actually, I'm surprised
that simply clicking "reply" isn't automatically interpreted as "reply
to list" when you click on an email from a list. It doesn't seem like it
would be that hard to do. Oh well.

- I can't use filters to put email into the junk folder. My inbound
emails include a Spamassassin "X-Spam-flag: YES" if they were scored as
spam by the ISP. I'd like to move these into the junk folder instead of
my inbox but I can't select "junk" when the move dialog asks for the
folder name. As it is, I had to create a "junk2" folder. The built-in
spam checker uses the junk folder, so now I have to go through two
different junk folders instead of one.

- I was surprised to see a bug frequently complained about in 1.4.x
still isn't fixed. The date formats displayed through the program are
wildly inconsistent. Even in the date column of the folder view, there
are many different date formats displayed at the same time. One email
shows a date of "Yesterday 02:00 PM", while the next shows a date of
"Fri 11:05 PM", scroll down a little and you'll see one that says "Sep
07 09:50 PM", and another that says "Aug 06 2003". Some have years, some
don't, some list the day of the week others don't. It's as if a random
number generate selects the date format. It would be much so much easier
to read if they all used a consistent format. And even nicer if the user
could select the date format. This is even more of a problem for
international users. Looking around in the GNOME bugzilla I found the
bug report I remember commenting on when I installed 1.4.x:

 http://bugzilla.gnome.org/show_bug.cgi?id=205137

But the loss of control over the local message store format is by far my
biggest complaint. The two-account kludge allowed me to get my mail
working but it's not an acceptable long-term solution. It will do until
I can find another email program that supports maildir or (even better)
until Evolution regains user selectable storage formats.

-Steve