Re: Occasional hangs - new info and ideas

[Jack <ostroffjh users sourceforge net>]

Trimming down, leaving only enough to perhaps tickle some memories:

On 2018.09.11 21:00, Jack wrote:
> > On 2018.08.28 15:13, Albrecht Dreß wrote:
> > > Am 28.08.18 20:57 schrieb(en) Jack via balsa-list:
> > [snip other issues...]
> > > > 3) For the past several weeks, I have been noticing an occasional,  
> > > > unpredictable hang/pause in Balsa when downloading messages.  […]   
> > > > I'm looking for any thoughts on how to troubleshoot this better.

> > > You might want to run Balsa with all debug messages  
> > > (G_MESSAGES_DEBUG=all) or only low-level networking messages  
> > > (G_MESSAGES_DEBUG=libnetclient) enabled and check if anything  
> > > unusual happens with the problematic mailbox.  Remember that  
> > > passwords will be printed in the output, so please sanitise any  
> > > files before posting them.
> > I've done this, but seen no messages that would indicate any  
> > particular problem.  However, the hangs mostly seem to be after  
> > Balsa finished downloading a message (often the last downloaded line  
> > in the DEBUG is a '.')  so I'm now wondering if the hang might be  
> > caused by checking gpg signatures - and like magic, I have had this  
> > issue in the past, and there was a thread in February about it.  You  
> > provided some notes on checking gpg configuration, which I'll have  
> > to read and follow up on again.  I have 504 "pub" keys (from "gpg  
> > -k") and I wouldn't consider that a particularly large keyring - or  
> > am I wrong?  (Browsing through the list of keys, I suspect most got  
> > there by due to messages from various mailing lists.)
> >
> > However, two more questions:  First, as I said in February, why  
> > would this check (or whatever is currently happening) completely  
> > block all Balsa activity?  Second, in my Inbox, I do have "Decrypt  
> > and check signatures automatically" set to "Never" so it's obviously  
> > not exactly the same problem I was having back them.
> >
> > Anyway - I've updated several bits of my gpg configuration, and I  
> > know I've got lots to go, but even if I'm imagining it, I seem to be  
> > having less of the hangs, but not totally gone yet.
> OK, I've now run with G_MESSAGES_DEBUG=crypto and seen the hangs with  
> no crypto related output at all to console.  Does that exonerate any  
> gpg stuff?  If so, where do I look next?
Right after my last reply, I cleaned up several issues with my gpg  
config, so I'm now more convinced that is not the problem.  I'm now  
wondering if it might be related to scanning an applying filters.   
However, I recompiled balsa with profiling support, and have the grpof  
output of one very long run (several hundred messages - returning from  
almost a week travel).  I'm not at all sure I'm correctly interpreting  
the output, but it looks like libbalsa_utf8_strstr is taking the most  
time, and it is called by message_match_real, called by  
libbalsa_mailbox_message_match, called by  
lbm_run_filters_on_reception_idle_cb.

So, I have lines of questions.  First, does anybody have more  
experience interpreting the gprof output?  If so, the gprof output is  
about 5000 lines, so probably not suitable for posting to a message  
here (does the mailing list allow attachments?).  I'll be glad to post  
here or two send individually.

Second, I assume "cb" is for callback - which implies that the filters  
are called every time a message is added to the mailbox.  If this tries  
to filter every message in the mailbox, instead of just the new one,  
that might be something to consider changing, although I can imagine  
times it would be appropriate to filter the entire mailbox.  The other  
option would be to only call the filters after all new messages have  
been added.  I know if I'm retrieving lots of messages, I sometimes  
read some in the mailbox to which they have been filtered, before all  
messages are retrieved, but I think I'd be willing to wait (or just  
read in the inbox) if it would stop these freezes.  As part of this  
question, though, it seems to be specifically libbalsa_utf8_strstr  
taking all the time - so I wonder if either it could be optimised, or  
if there is something funny about my configuration which is causing a  
problem others do not see.

For some additional info about the freezes - when it happens, the balsa  
itself is totally unresponsive.  I can adjust the window size (which I  
assume is done by the window manager, not balsa itself) but balse  
itself waits another many seconds before redrawing the window to  
reflect the new size, or to reflect if I've clicked on another mailbox  
or another message.  Even the "Checking Mail..." freezes - I cannot  
move it, and the sliding bars under each server being checked stop  
moving.  These freezes can last up to a minute or so, and seem to  
restart with each retrieved message, which seems to support my theory  
above.

Sorry to be so long winded, but I'm really hoping to provide enough  
information to either identify the problem, or to let someone give me  
some ideas on further troubleshooting.

Thanks.

Jack