Updating a TextView with the results of a long, processor-intensive process

[Bob Wilkinson <bob fourtheye org>]

I have a callback which processes a directory of files. The process_file
routine typically takes a reasonable amount of time, and is processor-intensive.

e.g. 

sub on_process_dir {
  my $d = new DirHandle $::settings->{'process_dir'};
  if (defined $d) {
    while (defined($_ = $d->read)) {
      next if (/^\.\.?$/);
      next if ($filter and not /$filter/);
      my $infile = join "/", $::settings->{'process_dir'}, $_;
      $::widgets->{buffer}->insert($::widgets->{buffer}->get_end_iter, "Processing $infile");
      process_file($in_file);
    }
  }
}

I have read the FAQ Q: How do i keep my gui updating while doing a long file read? 
and added code to the process_infile routine.

open ($fh, "perl ./process_signal.pl $input_file $output_file $params |") or die "can't fork";

my $tag = Gtk2::Helper->add_watch($fh->fileno, 'in', sub {
             if (eof($fh)) {
               Gtk2::Helper->remove_watch ($tag);
               close($fh);
             } else {
               my $line = <$fh>;
               $::widgets->{buffer}->insert($::widgets->{buffer}->get_end_iter, $line);
             }
             return 1;
});

However, it only updated my TextView at the end of the processing.

I also discovered, via ps ax, that each file was being processed 
simultaneously. I was testing with 2 files in the directory - though
could imagine problems were I to try to process many files, due to
the processor-intensive nature of the processing. The solution of
forking n subprocesses isn't scalable.

So I would like to be able to process the files sequentially - I have
found that if I build a list of files in the callback and pass this to 
the processing routine, I can effect this, though with the side-effect 
that the GUI is unresponsive, and the TextView still only updates when 
all of the files are processed. I would like the TextView to be updated
in real time, or at very least, on a file by file basis.

I am thinking that I need a more sophisticated approach. I could 
implement this via the use of the "threads" and "threads::shared" modules.
I have also found POE::Loop::Gtk2; I am not however sure of the best
approach, so am asking for guidance from any one with experience of using
either of these approaches.

If I have missed some better way to approach this problem, I would
welcome pointers to solutions.

Bob
-- 
Generally speaking, the Way of the warrior is resolute acceptance of death.
                -- Miyamoto Musashi, 1645

Attachment: signature.asc
Description: Digital signature