On Mon, Jul 13, 2009 at 3:55 PM, Martin Nordholts<enselic gmail com> wrote: > Always building the tests ensures that they are kept up do date with changes > in the rest of the code base. If they are not built along with the rest of > the code base then they will eventually become out of sync and someone has > to fix them later, possibly months after the change that introduced the > compilation errors were made. > That argument has essentially no value at all to me, for lots of reasons. First of all, compiling all tests while I'm experimenting on the code and nowhere near a checkin is just annoying. Next, just compiling the tests and not actually running them is almost useless, as it only finds API breakage. You need to actually run the tests to find the bugs. And in the context of glib there's a multiplied effect, as API breakage is found almost immediately. Then, I expect someone to run make check before doing checkins anyway. And if people forget it, there should be a build farm that catches it and complains loudly (either on IRC or on mailing lists), flagging the commit that broke. And finally, it's trivial to figure out which checkin broke a test with git bisect run. Note that I'm not opposed to having a useful test suite in glib and gtk - in fact, I'd welcome it. But it's important that the test suite not just aids in finding bugs but also has a very limited impact on hackers, so they don't outright reject it. The current setup seems to not be very good at that, as the number of tests does not seem to grow and people seem to never run it (yes, that includes me). I'd even say they are annoyed by it (see this mail thread), but that might just be me. If you wanted to improve the testsuite situation, you should have a look at other projects that successfully make heavy use of testsuites and copy what they do. Examples of such projects I've contributed to are Mozilla, Webkit or Cairo. (If a jibe is allowed: None of them build the testsuite on make, as it would at least double build times.) I've attached a patch that seems to work for me. Cheers, Benjamin
Attachment:
diff
Description: Binary data