Re: Testing for memory safety issues with Address Sanitizer

On Tue, 20 Sep 2016 23:02:54 -0700
Philip Withnall <philip tecnocode co uk> wrote:

It’s something which would be good to have running as part of CI,
along with the other -fsanitize options (address, thread, leak,
undefined) and things like -D_FORTIFY_SOURCE=1.

I don't want to discourage you here, but from all the sanitizers asan
is the one you most likely want to start with. It is really easy to use
and gives you the most significant bugs for the least amount of
worries. leak sanitizer is no sanitizer of its own, but part of the
newer versions of asan (available since gcc 5 I think). But not
everyone wants to fix every single leak and they are often harmless, so
it has to be weighted whether to focus on that (it can be disabled with
the env var ASAN_OPTIONS="detect_leaks=0")

ubsan (undefined behavior) is also easy to use and can be combined with
asan. However it also finds a lot of minor issues (mostly invalid shift
operations) that are more cosmetic.

msan and tsan (memory, threads) are significantly more bothersome to
use. I haven't done a lot with tsan. with msan the challenge is that
you need to compile *everything* with it, including all libraries. If
you use c++ even parts of the libc. It also has a couple of pitfalls
and you easily end up getting bogus false positives if you do anything

It would of course be nice if one used all sanitizers and fixed all
bugs, even the minor ones. But for practicality I'd say start by using
asan. It will give you more results for less work.

(also regarding fortify source: that's already the default on many
linux systems, e.g. ubuntu, therefore I assume testing this is already
practically happening. And it doesn't play well with the sanitizers,
it's best to disable it when using asan with -U_FORTIFY_SOURCE)

Enabling asan in the jhbuild configuration by default would be one
option, with the idea that people would then develop using it by
default. However, asan interferes with Valgrind, and I think that
Valgrind is probably the more useful tool to have working in a
development environment like jhbuild.

Be aware that asan basically replaces a lot of valgrind's
functionality and can find bug classes that a runtime tool  like
valgrind can't find by design (e.g. stack buffer overreads). The only
thing I'm aware what valgrind finds and asan doesn't are uninitialized
memory reads (there's msan for that, but see above, harder to use).

I should reiterate that I think doing testing like this manually is a
trap. For this to be useful, it needs to be automated — that will
rely on each module having good test coverage, but that’s a good goal
to aim for anyway.

I'm not familiar with gnome's testing strategies, but I'd assume it's a
very different thing running a test suite versus running the actual
application. In order to automate that one would need some kind of
simulated gui where simulated user interactions happen.

I still think it'd be valuable simply being able to run the whole gnome
stack on asan and see what bugs crop up. In my experience gui
applications are especially prone to use after free bugs (which may be
very significant in increasing stability, as these are the bugs that
you may hate as a developer - they may ocassionally cause crashes or
malfunctions, but not in any reproducible way).

Hanno Böck

mail/jabber: hanno hboeck de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42

Attachment: pgpBtE_wNrpiy.pgp
Description: OpenPGP digital signature

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]