Re: gtk2-perl / suggesting adding automated testcases

[Dermot Musgrove <dermot glade perl connectfree co uk>]

Hi all,

Guillaume Cottenceau wrote:
> goran kirra net writes:
>
> > I think you're on the right rack here.
> > The test suit used by a CPAN install:
> > bash# perl Makefile.PL && make && make test && make install
> > should run without requiring user interaction.
> > An interactive test suit run should be recommended but not required.
I guess that there are two sorts of tests that we want to run.

1) Those designed to visit _every_ corner of a feature - like testing
each and every enum in enums-as-strings.pl for a developer.

2) Those that test a Gtk2-Perl build - like testing that the enums 
generated by enums-as-strings.pl are available. This would only require
testing one or two enums and could be done as part of a more general 
widget test.

> Maybe it's then better to have a separate directory for automatic
> tests?
AFAIK the usual 'perl Makefile.PL && make test' will run all scripts 
called './t/some_file_name.t' Scripts in other sub-dirs are ignored
and so are files in ./t/ that don't have the suffix '.t' so it is
possible to have ./t/enums-test.pl which is available to developers
but will not be run by 'make test'. 

A package tarball would omit the developer's scripts by removing them
from the MANIFEST but CVS users would still download all the tests.

However, you might still feel that it is better to have separate 
directories for the developer's tests and the build tests.

> Or maybe all automatic tests might use the Gtk2::Test engine by
> Dermot. But I'm not sure it be versatile enough to let me do the
> eval {} test of enums-as-strings.pl, Dermot?
The tests that you suggested in your previous email could be run in
the normal testing in several ways - I think :)
1) in the @test_plan as:
  [$USE "Gtk2::Window"],
      ["new('toplevel')"],
      ["new('topleel')"],

2) or as manual tests:
  ok(try_use("Gtk2::Window")
  ok(try_new("new", 'toplevel'), $OK, $err);
  ok(try_new("new", 'topleel'),  $FAIL, $err);
  ok($err =~ /FATAL: invalid GtkWindowType value topleel/, $NOCHECK, $err)

I haven't tested this but something similar should work. 

3) I have changed Gtk2::Test so that you can specify $EXPECT_ERROR and a
check to apply to the error raised. In this case Gtk2::Test will compare
to $err (error message) rather than $ret (value returned by method). 

Obviously it would return $OK if the check is true but $err will still
contain the error and some explanations.

Then the tests would be 
  [$USE "Gtk2::Window"],
      ["new('toplevel')"],
      ["new('topleel')", $EXPECT_ERROR, 
          "=~ /FATAL: invalid GtkWindowType value topleel/"],

Since Gtk2::Test prepends some information to the error message, you
cannot expect the Gtk2 error to be at the start of the string.

Does this do what you need Guillaume?

Regards, Dermot