On Tue, 2017-02-28 at 18:05 -0600, Michael Catanzaro wrote:
Ah yeah, don't cancel your plans for nautilus. Regarding coverage. Most of our modules are core modules; we have a lot of them. I don't think we have the resources right now to write tests and obtain high coverage for more than a couple of these modules, unfortunately. I'd like to keep the GNOME goals manageable and achievable. You should still go ahead and add coverage support to nautilus, though. It's an excellent way to improve quality.
I agree that developers need to be engaged with adding more unit tests and code coverage if such a goal is to be useful. I wonder if making it a goal would kick-start some people to do that? I don’t think we can ever expect the majority of maintainers to care about (or have enough time to care about) code coverage and unit testing — but GNOME goals have been useful catalysts in the past. I guess a suitably well publicised and tutorialised blog post would work just as well though.
Regarding installed tests. The benefits of installed tests versus make check tests are not very clear to me. I don't think it should be necessary to install the tests in order to be able to run and test applications. That indicates a failure somewhere in the test infrastructure.
The comparison of installed-tests and `make check` is given on the goal page: https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests#Issues_wit h_.22make_check.22 Briefly: • installed-tests give a more consistent test environment, eliminating spurious test passes or failures due to differences in developers’ laptop configurations • installed-tests allows reverse-dependency testing: find test failures from new versions of libraries your project depends on, without having to rebuild your project (useful in a CI environment) • Much easier to integrate into a system like gnome-continuous • Allows for integration tests as well as just unit tests Importantly, there seems to be a misconception that it’s “installed- tests *or* `make check`”. That’s not true. Adding support for installed-tests to a module doesn’t mean `make check` goes away or becomes less useful. I’m strongly in favour of adding installed-tests to our modules. With the glib-tap.mk helper file, it’s pretty trivial. Before accepting it as a goal, though, the wiki page should be updated to give a howto on adding installed-tests support to a module. This is the commit I did for libgdata a while ago. It’s probably not perfect, but it’s an indication of what needs to be done: https://git.gnome.org/browse/libgdata/commit/?id=802fad78 Note that installed-tests also fits in nicely with distribution testing efforts like Debian’s autopkgtest: http://packaging.ubuntu.com/html/auto-pkg-test.html i.e. It’s a case of adding the following two files to a package: https://git.apertis.org/cgit/rhosydd.git/tree/debian/tests/control https://git.apertis.org/cgit/rhosydd.git/tree/debian/tests/gnome-deskto p-testing Philip
Attachment:
signature.asc
Description: This is a digitally signed message part