Re: speed issues with large mailboxes

[Olaf Frączyk <olaf cbk poznan pl>]

On Tue, 2003-07-29 at 12:59, emmanuel ALLAUD wrote:
>  --- Mpiktas <vaze0348@uosis.mif.vu.lt> a écrit : >
> Hi,
> > 
> > I'm using Balsa for quite a long time now and it's a
> > great mailer. Yet 
> > handling of large mailboxes (2000+ letters) is a bit
> > slow. For example on my 
> > Debian Sid, Athlon 750Mhz, 256MB RAM machine opening
> > of mailbox with 2800 
> > letters takes about 20 seconds, and after first 5-10
> > seconds Balsa becomes 
> > unresponsive till it opens the mailbox. Is there any
> > plans for improving the 
> > speed? Or is there some bottleneck which will slow
> > the process no matter how 
> > hard you will try to improve it? If there is such a
> > bottleneck, maybe it is a 
> > good idea to display say first 100 letters and load
> > others in background?
> > 
> > Mpiktas
> > 
> 
> Indeed there is an issue with large mailboxes. For now
> we use libmutt (which is mutt rearranged as a library)
> which IIRC parses the whole mbox file when you open a
> mailbox. To fix that we could only parse "partially"
> the mailbox, but this means that we have to rewrite
> part of the code if not all; but as you notice balsa
> is still responsive during this huge loading (it is
> done in another thread than the one controlling GUI),
> so it is not that bad. So the actual problem is the
> way we load the gtktreestore/view at once with all
> mails. In short we could perhaps just try to store the
> mails "on-demand", ie for instance try to parse the
> messages following the sort asked by the user and
> select the last ones (~100) display them, and then via
> a callback we should go on storing the remaining ones
> in the background. I don't know exactly the problems
> this approach would involve but AFAICT this kind of
> hack would certainly require a lot of changes so IMHO
> this would be better done in a devel branch. Pawel ?
> Bye
> Manu
> 
Hi,

This problem is known for very long time. Unfortunately I had to stop
using balsa because of this (I often have 3000+ messages in mailboxes).
When I was testing it (don't remember exactly - a year ago or more)
gtk-1.2 version - the slowest part was inserting into
gtk_clist/gtk_ctree. 
As gtk 2.x gtk_tree_view is even slower, I think that it is still
problem in this area.
I suggest to look into "pan" (newsreader) code, as they have very fast
GUI part. Both for gtk 1.2 and 2.x. AFAIR for 1.2 they were inserting
rows into gtk_ctree in some special order, and it was really fast (tens
times faster than balsa).
Additionally, the sorting was also very slow in balsa (with threading it
was real pain) - "pan" has also much faster threading/sorting.

Regards,

Olaf Fraczyk