Re: Updating a TextView with the results of a long, processor-intensive process

["muppet" <scott asofyet org>]

Bob Wilkinson said:
> On Tue, Apr 13, 2004 at 11:17:21PM -0400, muppet wrote:
> > On Tuesday, April 13, 2004, at 05:23 PM, Bob Wilkinson wrote:
> >
> > > I have read the FAQ Q: How do i keep my gui updating while doing a
> > > long file read?
> > > and added code to the process_infile routine.
> > <snip>
> > > However, it only updated my TextView at the end of the processing.
> >
> > Some sanity-check questions:
> > - Are you returning control to the event loop?
>
> Implicitly. once the processing routine is finished control returns.
> How would I do this explicitly, once the processing routine is
> started?

short answer:  either "Gtk2->main_iteration while Gtk2->events_pending;", or
restructure your code.


long answer:  there are several ways, and the choice depends on how hard it is
to adapt your algorithm to the method:

a) put the processing into a child process, communicating via a pipe; then
watch the pipe for events, and update accordingly in the io watch callback.  
this is what you're trying to set up.  (the worker thread model is similar to
this, except you have more communication options.)

b) break the processing up into little chunks and put the chunks into a work
queue.  in an idle handler, process the next chunk.  this is how the gimp
remains responsive while applying filters and updating its display.

c) at "Strategic points" in your heavy processing loop, process events from
the main loop.  this is a quick and dirty way to do it, and not really a good
design.  if tossing this sort of thing in fixes the problem, then you know
that you need to restructure your code to work like a) or b).


the key is to keep the event loop running as much as possible, which is why a)
is often best for really long-running and intensive processing --- the
processing is not worried about keeping things responsive, it just worries
about doing its job.

think of your callbacks as interrupt handlers --- they need to do their work
and get out as quickly as possible.  if you exceed your processor bandwidth,
the system becomes unresponsive.


> > - Is the child doing buffered writes which would result in a chunk at
> > the end even though it should print out other stuff in the middle?
>
> Not that I am aware of. The processing routine is an XS wrapper
> around a C library. The C library "printf"s percentage progress
> completed. If I run it from a terminal it appears unbuffered. Prior
> to me starting to show this progress in a TextView pane it was
> appearing, apparently unbuffered, in the terminal from which the GUI
> interface was started.

it should be possible for you to read the child's standard output, parse the
%age numbers, and update a progress bar.


> > You need to implement some form of throttling mechanism that allows at
> > most $n_active_children at a time; then sequential processing becomes
> > $n_active_children=1.  There are tons of ways to do it, using
> > semaphores, file locking, a counting reaper, chaining handlers, etc etc.
>
> I tried setting a global lock variable, and to run the children only
> when the lock was unset. Unfortunately they ignored me and ran
> anyway.

Remember that the parent and child can communicate only via IPC mechanisms ---
they cannot share a lock variable.


> > And then the queue handler is some object that knows about process
> > throttling and synchronization and all that.
>
> OK. But all of the GUI seemed insensitive while processing the files
> - they typically take about 20-30 seconds to process each file.

This means you're blocking somewhere, possibly in the filehandle read in your
callback.  Where does the code stop?  Either put in some tracer prints or run
in the perl debugger to find out.  Do you get out of the click handler?  Do
you get into your IO watch handler?  Do you get out of it?

I suspect you're blocking on the read in the IO watch handler.  THis:

               my $line = <$fh>;

is going to try to read until it sees a newline.  I typically do

               local $/ = undef;
               $text = <$fh>;

to avoid blocking there.  The minor disadvantage is that you have to handle a
multiline string, but that's not a big deal.



-- 
muppet <scott at asofyet dot org>