Unix Memory Management : Clearing up a few misconceptions (was Re: how can I trust glib when it has so manymemleaks?)

[Guillaume Laurent <glaurent worldnet fr>]

(Unix programmers with a clue can skip this one - or proofread it for
mistakes:-)

First of all, though a few people in this thread have already
correctly stated that free() does not release mem to the OS, even more
people still have this misconception that, when you free() malloc'ed
memory, it is released to the OS. THIS IS FALSE, under most
implementations. So, to state things once and for all (yeah right :-) :

UNDER UNIX, WHEN YOU FREE() MEMORY, IT IS NOT RELEASED TO THE OS. THE
SIZE OF YOUR PROCESS WON'T EVER SHRINK, IT CAN ONLY INCREASE.

This is a comp.unix.programming FAQ (read it for more details). Most
malloc()/free() implementation don't do that, because it's very slow
and very complicated to do (if at all doable : it would involve
repacking mem chunks and modifying the value of all pointers which
point to the ones you have moved), and most of the times isn't worth
it. What actually happens is that memory is kept for further malloc()
calls.

e.g.:

ptr = malloc(10 * 1024) ; // Your process size increases by 10K
free(ptr); // Your process size DOES NOT CHANGE
ptr = malloc(10 * 1024) ; // Your process size DOES NOT CHANGE either.

Some special malloc()/free() implementation *are* able to actually
release memory to the OS when needed (I believe XEmacs uses such a
thing), but this is a very special case.


About malloc()ating pools of memory :

This is mostly useful to palliate slow malloc() implementations for
programs which use it heavily. With this mechanism, a malloc() then
roughly amounts to increasing a pointer.

Nowadays most malloc() implementations don't really suffer performance
problems. However, even the best one still involves actually
requesting memory to the OS (with sbrk()) and this will always be
slower than merely increasing a pointer. So the glib mem pool
mechanism is certainly worth it, given how heavily GTK+ relies on
malloc().


About memory leaks :

The OS gets the memory back in one swift move at the death of the
process, whether you free()d it or not. If your program isn't supposed
to run for long periods of time, then you probably don't care about
leaks. However, sloppy programming will always come back and bite you
(hint: bad habits, or when you'll need your program to run for such
long periods of time).

Stricly speaking, the definition of a leak may vary. It can either be
memory that you haven't free()d (a bit too wide a definition IMHO) :

e.g.

ptr = malloc(10);
// do stuff, don't free ptr

but this kind of "leak" is by definition very much bounded,

or it can be memory that you *cannot* free() anymore, because you lost
the last pointer to it :

ptr = malloc(10);
// do stuf, don't free ptr
ptr = malloc(10); // you leaked 10 bytes

This is of course the only really dangerous kind of leak, especially
when it occurs in a loop, and programs which run for a long time
usually have many loops.

However, it's hard for a malloc debugger to make the difference
between a bounded leak and a real leak. Some are able to do so, but it
requires heavy wizardry, that is keeping track of the pointers
pointing to mem chunks, rather than the mem chunks themselves. So
that's one more reason for you to properly free() what you malloc(),
or be prepared to sort through the results of your malloc debugger.

And indeed C++ (and OO programming in general) does make this kind of
issue very much easier to deal with, thanks to the ctor/dtor paradigm
(but brings out a couple more which are just as funny).

-- 
					Guillaume.
					http://www.worldnet.fr/~glaurent