Re: Updating a TextView with the results of a long, processor-intensive process

[muppet <scott asofyet org>]

On Tuesday, April 13, 2004, at 05:23 PM, Bob Wilkinson wrote:

> I have read the FAQ Q: How do i keep my gui updating while doing a 
> long file read?
> and added code to the process_infile routine.
<snip>
> However, it only updated my TextView at the end of the processing.

Some sanity-check questions:
- Are you returning control to the event loop?
- Does the child print anything out before the end of processing?
- Is the child doing buffered writes which would result in a chunk at 
the end even though it should print out other stuff in the middle?
- Would a terminal emulator widget, e.g., Gnome2::Vte actually make 
more sense for this?


> I also discovered, via ps ax, that each file was being processed 
> simultaneously. I was testing with 2 files in the directory - though 
> could imagine problems were I to try to process many files, due to the 
> processor-intensive nature of the processing. The solution of forking 
> n subprocesses isn't scalable.

I presume this is because the parent forks and goes on; from the looks 
of your click handler, you spawn a child for all the entries in the dir 
before returning from the click handler.

You need to implement some form of throttling mechanism that allows at 
most $n_active_children at a time; then sequential processing becomes 
$n_active_children=1.  There are tons of ways to do it, using 
semaphores, file locking, a counting reaper, chaining handlers, etc etc.

An "interesting" side effect of such a mechanism is that the button 
click will actually start the processing, not contain all of the 
processing.  You'll want something like

  on click
     scan dir to create list of files to process
     add these files to the processing queue
     start the queue
     desensitize portions of the ui that the user shouldn't bother 
while you're busy
     sensitize a "cancel" button

And then the queue handler is some object that knows about process 
throttling and synchronization and all that.


> So I would like to be able to process the files sequentially - I have 
> found that if I build a list of files in the callback and pass this to 
> the processing routine, I can effect this, though with the side-effect 
> that the GUI is unresponsive, and the TextView still only updates when 
> all of the files are processed.

That sounds a lot like control is not returning to the main loop until 
the job is finished.  What other stuff is happening in process_file()?


> I am thinking that I need a more sophisticated approach. I could 
> implement this via the use of the "threads" and "threads::shared" 
> modules.  I have also found POE::Loop::Gtk2; I am not however sure of 
> the best approach, so am asking for guidance from any one with 
> experience of using either of these approaches.

Even if you used POE, you'd still have to return control to the main 
loop in order for the UI to be responsive, and that will require some 
form of asynchronous processing.

I can't recommend threads in good conscience, for several reasons:
- A fork/pipe system will be a little more stable and robust (on unix, 
at least).
- Using threads requires a threaded perl, which is not available 
everywhere, whereas perl's forking open() is.
- You can't share widgets with threads::shared.  In fact, even with the 
Gtk2::Gdk::Threads functions (which don't appear in the FAQ, i notice), 
there are "interesting" issues involving Glib::Objects being destroyed 
out from under the main thread when a worker thread exits, for which 
there are no clean solutions.

--
That's it!  It's one thing for a ghost to scare my children, but it's 
another to play my theremin!
  - Homer Simpson