Glib::IO->add_watch and reading the filehandle

["Matthew Braid" <ptkperl mdb id au>]

Hi all,

I've just run into a little problem in one of my programs involving a
Glib::IO watch.

In order to allow the main program to run smoothly while Big Stuff(tm)
is happening, I've shifted a lot of time intensive work off to a
helper process which has been opened with IPC::Open2. Messages are
sent to the helper process and responses are picked up by the Glib::IO
watch. Usually this works fine with the following callback:

sub {
  my ($fno, $cond, $self) = @_;
  if ($cond & 'in' and
      sysread($self->{FromChild}, my $buff, 1024)) {
    $self->recvchunk($buff);
  }
  if ($cond & 'hup') {
    $self->recvchunk("\n");
    $self->{PipeOK} = 0;
    return 0;
  }
  return 1;
}

(to clarify, $self is passed in as data to the callback and is an
object that handles incoming and outgoing messages, translating as
needed. recvchunk is the sub that handles incoming blocks of data, and
$self->{FromChild} is the filehandle for the incoming messages).

Every so often it would stop working (ie, the helper process would
claim it sent a message, but the main process wouldn't act on it), and
after a lot of debugging it turned out that if an incoming message was
longer than 1024 characters the remaining part after the first 1024
characters wouldn't be handled until the _next_ message was sent - in
other words, if I sysread 1024 characters off a 1025
character-available stream, the Glib::IO watch doesn't still see the
file handle as readable, and so doesn't trigger the callback until
more data is pushed onto it. Unfortunately I can't use a while
(sysread(...)) in the callback cos then it hangs waiting for data.

As a workaround I upped the sysread to 10240 characters (which is
very, very large for a message in this system), but I'm pretty sure
this isn't Behaving As Expected, so I wanted to make sure I haven't
done something stupid in my callback.

Since I control both sides of the communication, I _can_ ensure that
the helper script never sends a message exactly 1024 characters in
length (empty lines are ignored, so sticking a newline at the end of a
1024 char message will work fine) and change my callback to something
like:

sub {
  my ($fno, $cond, $self) = _;
  if ($cond & 'in') {
    while (1) {
      my $cnt = sysread($self->{FromChild}, my $buff, 1024);
      $self->recvchunk($buff);
      last if $cnt != 1024;
    }
  }
  if ($cond & 'hup') {
    $self->recvchunk("\n");
    $self->{PipeOK} = 0;
    return 0;
  }
  return 1;
}

but that only works _because_ I have full control on the incoming data
and isn't really a generic solution.

In case its important, messages sent back and forth are always
hashrefs that have been serialised into YAML and then Base64 encoded
and wrapped in '--- START MESSAGE---' and '---END MESSAGE---' lines,
so a full message is never a single unbroken line. Filehandles have
also been unbuffered with select.

TIA,
MB