Re: GLib test framework for your own project

[Tim Janik <timj imendio com>]

On Wed, 9 Jan 2008, Tommi Komulainen wrote:

> Hi,
>
> Here's a quick guide for setting up GLib testing framework for your own
> project. It is the result of some trial and error when integrating for
> hildon widgets the test framework from current trunk. There are some
> autotools related details that might not be immediately obvious for mere
> mortals. Thought it good to collect the details in one place. Hope it
> helps.

thanks for doing that.

> 1. Copy Makefile.decl
>
>        [glib or gtk+ currently does not install any usable
>        Makefile.decl]

yeah, i was thinking about that when i wrote it.
once the current testing rules have stabelized and turn out to be
actually useful for testing glib/gtk+, it could make sense to install
a Makefile.gtkrules or similar for other projects to include that
provides the rather complex test framework rules. 
it'll be a bit tricky to include that though, given that the include
paths may vary. an alternative could be to provide the rules in an
automake macro, setup by AM_PATH_GTK_2_0().

>        Copy Makefile.decl from glib or gtk+ source directory to the
>        root directory of your project. The difference is that
>        Makefile.decl from gtk+ will run the tests in Xvfb and thus only
>        works for x11 backend.

right, for non-X11 targets, the tests will currently be skipped.
if people can provide alternatives to Xvfb for non-X11 platforms to
hide GUI program execution, that'd be much apprechiated.

>        Edit Makefile.decl so that GTESTER and GTESTER_REPORT are
>        correct for non-GLIB packages:
>
>        GTESTER        = gtester                # in $PATH for non-GLIB packages
>        GTESTER_REPORT = gtester-report         # in $PATH for non-GLIB packages
>
>
>        When using Makefile.decl from gtk+ also add the following line
>        after GTESTER_REPORT. This is needed for properly starting Xvfb
>        for the test.
>
>        gdktarget := $(shell pkg-config --variable=target gdk-2.0)

good point. however $(shell) is a GNU-make-ism, so for projects that
need to be portable across GNU-make, it might be good to provide
@GTK_GDKTARGET@ by AM_PATH_GTK_2_0().

> 2. Edit every Makefile.am
>
>        Add the following line to the top of every Makefile.am. This is
>        needed to enable recursive 'make test' and friends.
>
>        include $(top_srcdir)/Makefile.decl

one word about recursion here.

gtk+/Makefile.decl currently ignores and won't recurse into subdirs
with the names "po" or "po-properties". this may or may not be appropriate
for other projects (it usually is though, because po/Makefile.in is usually
not under the project maintainers control, and so is hard to extend for
additional recursive rules).

> 4. Write src/tests/testwidget.c
>
>        Make sure your normal build flags do not include -DG_DISABLE_ASSERT
>        (or add #undef G_DISABLE_ASSERT at the top of the file) as it will disable
>        all checks using g_assert() -- though not any of g_assert_cmp*()
>
>        [See some other tutorial and API reference for how to organize
>        the C code and which functions/macros to use. Currently glib
>        trunk does not generate any documentation for the testing
>        framework.]

hm, right. the API docs are there in glib/glib/gtestutils.c, but nothing
is generated for the web atm. will have a look.

> 5. Run the tests
>
>        $ make test
>
>        When running in emacs/vim the first failing test should be
>        recognized and the cursor should be placed on the line of
>        failing assertion.
>
>        [See some other tutorial/reference for how to run tests more
>        fine grained (./testwidget -p /foo, ./testwidget --help?)]

gtester --help works. test programs currently do not provide a --help
output, because they are programs in their own right. they do interpret
a bunch of options though, such as --g-fatal-warnings, -p=path, -m=mode,
-q, --quiet, --verbose, -l, --seed=randomseed, all of which correspond
to the respective gtester options, described by gtester --help.

given that g_test_init() parses all these args and that test programs
are unlikely to implement sophisticated argument parsing and --help
themselves though, it's probably best to hard code --help output into
g_test_init()...

> --
> Tommi Komulainen                            <tommi komulainen nokia com>

---
ciaoTJ