Re: [Evolution-hackers] Performance with Exchange 2003

[Lee Revell <rlrevell joe-job com>]

On Thu, 2005-04-28 at 09:59 +0530, Not Zed wrote:
> On Wed, 2005-04-27 at 02:28 -0400, Lee Revell wrote: 
> > On Wed, 2005-04-27 at 10:02 +0530, Not Zed wrote:
> > > On Wed, 2005-04-27 at 00:22 -0400, Lee Revell wrote: 
> > > > On Wed, 2005-04-27 at 09:26 +0530, Not Zed wrote:
> > > > > A search for "" is alway optimised anyway, it doesn't actually perform
> > > > > any comparisons.
> > > > 
> > > > Then where does the *massive* speedup come from?  Have you actually
> > > > tried my patch?
> > > 
> > > I haven't so far; for starters it doesn't maintain all the
> > > functionality.  But I did try to recreate the slowness you were fixing
> > > - so i had something to compare to.  I can't really see it myself, it
> > > doesn't seem any slower changing to the unread vfolder vs any other
> > > folder, and most of the time doing either is building the nodes for
> > > the etreemodel.
> > > 
> > > I guess you must have some other option set that I don't, which is why
> > > i'm not seeing this speed problem.  e.g. hiding deleted messages, or
> > > some other view option.  Or a particularly large message set? (>200K)
> > > Or lots of folders? (>100)  Or a slow machine/with little memory?
> > > 
> > 
> > I do have a slowish machine (600 MHz) combined with a large message set
> > (~110K messages) and about 50 folders.
> > 
> > I have a patch that retains the full functionality by making "hide junk"
> > configurable, but it's not quite ready to post.
> I don't think we want hide junk configurable - it is just ui clutter,
> and there is no way to differentiate junk messages in the ui either.
> You'd have to ask UI (e.g. dobey) for a firm decision on this though. 

Makes sense.  I only suggested it to demonstrate the problem.

> > The maximum speedup will be seen by disabling "hide junk" and "hide
> > deleted" combined with my search bar fix.  With this combination, I
> > expect the difference to be noticeable even on the fastest machines.
> Well, you can't disable 'hide junk', so that is a strange test to
> suggest.  But from timing I got yesterday it might make a 2x, perhaps
> 3x speedup at most.  Not sure that's 'massive', but it is certainly
> significant.
> 

You are confusing 2 separate performance issues.  Maybe I didn't explain
it clearly.

One is that "hidejunk" is implemented suboptimally, via the folder
search.  This is a relatively minor issue, I could not measure it, but
2x-3x sounds about right.  I would expect this one to disappear into the
noise on anything beefier than, oh, an Athlon XP.

The other, MUCH worse, is that searches for "" are not properly
optimized away.  With my config (slow machine, plenty of RAM but CPU
starved, and 110,000 messages), this results in probably a 100-1000x
performance hit, truly massive.  I am really surprised that your
machines are all so fast that this one isn't apparent.  What's the
slowest thing you use to test for performance regressions?

> It seems like an easier fix would just be to do the hide deleted/hide
> junk code manually, you could do the whole lot with one binary test in
> a very small loop.  And remove it from being a search term.  That will
> run very fast, and i think you can re-order the uids (i.e. use
> g_ptr_array_remove_fast), so it will also be space efficient.  It will
> simplify the code too. 

Excellent idea.

> > I can see the code section that 'optimizes away searches for ""' but you
> > still burn a *lot* of CPU parsing headers before getting to that point.
> > My profiling showed a heck of a lot of time in strncasecmp() with a null
> > search, which I then verified by code inspection, and with the debugger.
> 
> Don't have a backtrace by any chance?  I'd like to see why it is doing
> that.  Even if this can be worked around by removing unecessary
> searches, it means the code is doing more work than it has to
> sometimes. 

I'll get one for you today.

Lee