Re: Testing for memory safety issues with Address Sanitizer


On Mon, 2016-09-19 at 12:03 +0200, Hanno Böck wrote:
I recently reported a couple of bugs to GNOME-components that were
easily discovered with a feature called Address Sanitizer (asan): (glib) (glib) (gnome-session)

The first two are out of bounds reads in glib that get triggered by
running the test suite with "make check". The third is a buffer
overflow in gnome-session that gets triggered on every startup.

Nice work!

Things that could be done, by increasing complexity:

* Run the existing test suites with address sanitizer
This could be mostly automated, it's usually as simple as doing
something like:
./configure CFLAGS="-fsanitize=address -g -fno-common
-U_FORTIFY_SOURCE" CXXFLAGS="-fsanitize=address -g -fno-common
-U_FORTIFY_SOURCE" LDFLAGS="-fsanitize=address -g -fno-common
ASAN_OPTIONS="log_path=/var/log/asan/asan-error" make check

It’s something which would be good to have running as part of CI, along
with the other -fsanitize options (address, thread, leak, undefined) and
things like -D_FORTIFY_SOURCE=1.

Unfortunately, these flags are mostly mutually exclusive, which means that we’d need 5 new CI pipelines.

I guess the right approach is to set up a few more parallel instances of gnome-continuous, each running with 
different compiler options. I’ll have a play with that tomorrow. I guess eventually it could be integrated 
into gnome-continuous as a new set of tree suffixes.

* Run GNOME itself and its applications with asan enabled.
Test suites are fine, but they don't replace real testing, i.e.
actually running the software.
Ideally one would built the full software stack with address sanitizer
enabled to find bugs both in the applications themselve and in

Automated tests complement manual testing, and require a lot less effort
over time. (While it would be lovely in theory, in practice, very few
maintainers are going to recompile each of their projects with 5 different
sets of compiler options and manually test every feature in the UI before a
release — that’s a huge amount of work.)

Enabling asan in the jhbuild configuration by default would be one option,
with the idea that people would then develop using it by default. However,
asan interferes with Valgrind, and I think that Valgrind is probably the
more useful tool to have working in a development environment like jhbuild.
Additionally, this means consciously prioritising asan over any other
-fsanitize or hardening flags (which is fine, but it is a tradeoff).

* Use Fuzzing.
This is especially interesting for everything that is parsing complex
data formats.
There has been a lot of improvement in fuzzing techniques lately and
advanced fuzzing tools like american fuzzy lop or libfuzzer can greatly
help in improving and securing code.

I have some introductions to these things on the fuzzing project

Not really related to asan, but good advice all the same. afl is pretty

I discussed this already with Tobias Mueller recently and he already
started finding a couple of issues with asan:

I should reiterate that I think doing testing like this manually is a trap.
For this to be useful, it needs to be automated — that will rely on each
module having good test coverage, but that’s a good goal to aim for anyway.


Attachment: signature.asc
Description: This is a digitally signed message part

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]