Re: Some shortcomings in gtestutils

hi Federico;

On 21 February 2013 02:46, Federico Mena Quintero <federico gnome org> wrote:

So, I've been learning what gtestutils provides.  It is not bad, but it
seems pretty pedestrian on some counts.  These are some things I'd like
to change, or at least for someone to point me in the right way of doing

* Tests abort as soon as a test fails, since tests ultimately depend on
variations of g_assert().  This is fine in that it makes the "ensure
things are okay" code look the same as "generic sanity check" code.
However, it also means that a test suite aborts as soon as it encounters
the first test that doesn't pass.  There is g_test_fail(), but the
documentation pretty much only says, "You can use this function if your
test failed in a recoverable way."  I don't know if that means that the
test couldn't find a data file (but can continue anyway), or if it means
"call this if you want the test to fail but want the test suite to
continue running".

Matthias, Colin, Dan, and I were discussing this in the bug to add TAP
support for GTest, to get decent report harnesses instead of our
homegrown test report XML hack:

in short, the only way to safely run these tests in C is to execute
each unit inside a separate process; it's not a trivial change,

for Clutter, we do compile all units inside the same binary (to cut
down compilation and link time), and then execute each unit
independently; we still abort() as soon as one unit asserts, but we
can easily change that and keep running. I also added a skip() and a
todo() wrappers, but that was just for my convenience.

* It's hard to get a debugger up in the middle of "make check".  I can't
think of a way to do it other than manually inserting code to sleep in
the middle of the faulty test, and then attaching with a debugger
manually.  Maybe having an environment variable or something, so that I
could run

  $ G_PAUSE_ON_FAILED_TESTS=1 make check

you can use TEST_ENVIRONMENT to set up the test environment variables
and command line arguments. it's standard automake. I use it to pass
the backend to the ABI check script, so that it's aware of which kind
of ABI should be available.



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]