Re: Some shortcomings in gtestutils

[Sam Spilsbury <smspillaz gmail com>]

On Thu, Feb 21, 2013 at 1:17 PM, Peter Hutterer
<peter hutterer who-t net> wrote:
> Having worked with googletest and xorg-gtest [1] for X integration testing,
> I can say the most annoying bit is to get the whole thing to compile. The
> C++ ODR prevents us from building gtest and xorg-gtest as library and then
> compiling against that. and autotools is not happy with having external
> source files. if you're planning to share a test framework across multiple
> source repositories, that can be a major pain.
>
> [1] http://cgit.freedesktop.org/~whot/xorg-integration-tests/

I agree, this is the one drawback of google-test.

> > > *snip*
>
> fwiw, one of the drawbacks I found with the multiple binary case is that it
> reduces the chance of running all tests every time. there's a sweet spot
> somewhere between too many and too few binaries and I suspect it differs for
> each project.

A good way to handle that is to have a separate test-runner that runs
the make check/test target. That can usually just be a makefile rule
or something else (in compiz we use ctest)

> for the googletest case for example separate binaries will give you a
> separate junit xml output, which make some regression comparisons harder.

We ran into this problem as well.

I think the solution was two fold:

 1. First of all, we wanted to be able to introspect test binaries so
that the test runner would be able to show each individual one. [1] is
a massive hack, but it works.
 2. Second of all, we had to patch google-test to shut up about
skipped tests in the junit.xml so that Jenkins didn't think you had
like 600,000 tests or something. I can provide this patch upon
request, its just somewhere in the midsts of the Canonical Jenkins
deployment.

[1] 
http://bazaar.launchpad.net/~compiz-team/compiz/0.9.9/view/head:/cmake/src/compiz/compiz_discover_gtest_tests.cpp

-- 
Sam Spilsbury