[Evolution] IMAP problems and workarounds

[Andrew Cowie <andrew operationaldynamics com au>]

Over the past few weeks, I experienced some show-stopping problems with
my IMAP in evolution 1.2.1 . This message summarizes two such problems,
and presents the workarounds that I discovered/implemented for each. 

The problems are

        1. Evolution hangs if it looses it's remote connection
        2. Evolution's hangs after interacting with UW-IMAPD

In particular, I suggest the workaround to the second problem be at
least added to a FAQ; further work on the root cause ("evolution's IMAP
code isn't very strong") would be magnificent and I encourage Ximian
management to support such efforts on behalf of all their IMAP users.

Before I start though, thanks to the entire development team for such a
wonderful email client. Evolution is [mostly ;)] a pleasure to use, and
I know that we all greatly appreciate the hard work.

AfC

--

1. Evolution hangs if it looses it's remote connection
------------------------------------------------------

BACKGROUND:

Sure enough, as Scott Otterson reported, Evolution hangs when if it
looses an IMAP connection. He was reporting it in the context of his
upstream ppp Internet link going down. I had also reported to this list
the evolution hanging when an ssh tunnel carrying an IMAP connection
dropped.

I access one of my client's office mail servers [Exchange as it happens]
via IMAP. In order to get through the firewall, I ssh to one of their
Unix machines, and as a part of that connection I port forward my
localhost:143 to the remote mail server's 143. By setting up an IMAP
server of "localhost" in evolution, I can access that mail.

[FEATURE REQUEST: Note the limitation that I can only pull this trick
once; if evolution allowed me to configure the IMAP port that would
allow me to have more than one remote IMAP client ssh-tunnelled in this
way and thus would not have to depend on IMAP-SSL, which is not
available to me in this case]

PROBLEM:

I discovered that after a period of time (ranging from 5-30 minutes)
suddenly evolution would be trying to retrieve a message, or rescan
folders or some other remote action, and would freeze there. Evo was
totally unresponsive, and required BOTH killev and oaf-slay before being
able to reconnect successfully on restart of evolution.

I noticed after a while that the ssh (terminal) sessions were hung as
well whenever this happened. Strangely, whenever I was *using* a shell
on this remote system, it never froze up - just when I opened an ssh
connection and left it there with tunnels. Tons of data pouring over the
tunnelled connection, but apparently ssh (+bash? Yes, TMOUT is unset)
seizes up if nothing is actually happening at the remote end, be that
usage or problem.


WORKAROUND:

I managed to improve my ssh connection stability by running a program at
the remote server - `top` as it happens. That successfully held the ssh
connection open and as a result Evolution stopped crashing. Yeay.


[As `top` is a bit much to keep running and wastes unnecessary
bandwidth, I wrote a trivial program which just sat there incrementing a
counter and sleeping for a second to use as the remote process.

That worked, but I discovered it was difficult to come up with a way to
automate killing the remote process when I shutdown evolution here on my
client side. In order to have a way to tell the remote program to exit
(and thus have the ssh session tear down cleanly) I augmented the
program to open a socket connection on a particular [remote] localhost
port, and wait for a connection. As soon as it receives a connection, it
simply exits. Then, as a part of setting up the ssh connection with the
tunnel to the IMAP server which runs this tiny program, I configure an
additional tunnel to the remote Unix machine targeted at the port this
program is listening on. To terminate the program (and hence shutdown my
ssh tunnel) I simply make a connection to my localhost:port and that
connection [attempt] is forwarded to the tiny program on the remote
server.

The program is called `keepalive`. If anyone else has a similar problem
I will gladly share it with them for further testing]



2. Evolution hangs after interacting with UW-IMAPD
--------------------------------------------------

BACKGROUND:

I had successfully been using Evo 1.2.1 against a Courier IMAP server  I
was running with no difficulty. However, a few weeks ago I switched my
mail (different account than the corporate one above) to a new colocated
hosting provider. Although I have IMAP access to my mailbox (which I
prefer over POP) I discovered that it they run the UW-IMAPD server. That
made me nervous as previous messages on this list discussed the UW IMAP
server and didn't say encouraging things about evolution's ability to
interact with it (even though it's the reference IMAP implementation!).

[Sorry, Jeff, but your earlier advice of "just tell your ISP to change
their IMAP server to courier" was not exactly realistic. Hosting
providers and ISPs do not magically reconfigure their infrastructure
just because one client out of thousands want them to. Given Ximian's
aim to make Evolution accessible to a wide ranging audience, that was
not a very constructive response]

PROBLEM:

As soon as I switched to my new hosting provider, I started noting
Evolution hanging again after 5-10 minutes of normal functioning.

It would be saying "Pinging IMAP server" and just sit there.


WORKAROUND:

Searching around the list archive and the FAQ I came across other
[unrelated] IMAP problems; in those cases the solution was to tell the
IMAP configuration to "check for new messages in all folders".

On a hunch, I tried turning on "Automatically check for messages every [
] minutes" on the same configuration panel, setting the interval to 10
minutes. No change. I didn't really expect much, as IMAP in maintaining
its connection, doesn't really have a notion of checking for new
messages in the way that one has to periodically with a POP3 box.
However...

I then tried dropping it to a shorter  (3 minute) and then 1 minute
interval between checks. Since I have instructed the IMAP configuration
to check for new messages every minute it has not lost its connection to
the UW-IMAPD server, and Evolution has not crashed. Yeay!!

[This one needs to go in the FAQ!]

--

[As previously reported, I am running Debian Sid/testing/3.1 with
Ximian's Debian Woody/stable/3.0 packages of Evolution 1.2.1 and
otherwise no problems]

I will enter either or both of these issues as bugs if so directed.

Regards,

        Andrew

-- 
Andrew Frederick Cowie
Operational Dynamics Consulting Pty Ltd

Australia +61 4 1079 6725  North America +1 646 270 5376

andrew operationaldynamics com au