On Tue, 20 Sep 2016 23:02:54 -0700 Philip Withnall <philip tecnocode co uk> wrote:
It’s something which would be good to have running as part of CI, along with the other -fsanitize options (address, thread, leak, undefined) and things like -D_FORTIFY_SOURCE=1.
I don't want to discourage you here, but from all the sanitizers asan is the one you most likely want to start with. It is really easy to use and gives you the most significant bugs for the least amount of worries. leak sanitizer is no sanitizer of its own, but part of the newer versions of asan (available since gcc 5 I think). But not everyone wants to fix every single leak and they are often harmless, so it has to be weighted whether to focus on that (it can be disabled with the env var ASAN_OPTIONS="detect_leaks=0") ubsan (undefined behavior) is also easy to use and can be combined with asan. However it also finds a lot of minor issues (mostly invalid shift operations) that are more cosmetic. msan and tsan (memory, threads) are significantly more bothersome to use. I haven't done a lot with tsan. with msan the challenge is that you need to compile *everything* with it, including all libraries. If you use c++ even parts of the libc. It also has a couple of pitfalls and you easily end up getting bogus false positives if you do anything wrong. It would of course be nice if one used all sanitizers and fixed all bugs, even the minor ones. But for practicality I'd say start by using asan. It will give you more results for less work. (also regarding fortify source: that's already the default on many linux systems, e.g. ubuntu, therefore I assume testing this is already practically happening. And it doesn't play well with the sanitizers, it's best to disable it when using asan with -U_FORTIFY_SOURCE)
Enabling asan in the jhbuild configuration by default would be one option, with the idea that people would then develop using it by default. However, asan interferes with Valgrind, and I think that Valgrind is probably the more useful tool to have working in a development environment like jhbuild.
Be aware that asan basically replaces a lot of valgrind's functionality and can find bug classes that a runtime tool like valgrind can't find by design (e.g. stack buffer overreads). The only thing I'm aware what valgrind finds and asan doesn't are uninitialized memory reads (there's msan for that, but see above, harder to use).
I should reiterate that I think doing testing like this manually is a trap. For this to be useful, it needs to be automated — that will rely on each module having good test coverage, but that’s a good goal to aim for anyway.
I'm not familiar with gnome's testing strategies, but I'd assume it's a very different thing running a test suite versus running the actual application. In order to automate that one would need some kind of simulated gui where simulated user interactions happen. I still think it'd be valuable simply being able to run the whole gnome stack on asan and see what bugs crop up. In my experience gui applications are especially prone to use after free bugs (which may be very significant in increasing stability, as these are the bugs that you may hate as a developer - they may ocassionally cause crashes or malfunctions, but not in any reproducible way). -- Hanno Böck https://hboeck.de/ mail/jabber: hanno hboeck de GPG: FE73757FA60E4E21B937579FA5880072BBB51E42
Attachment:
pgpBtE_wNrpiy.pgp
Description: OpenPGP digital signature