Re: [xml] Suggesting something like malloc_trim after freeing

On Fri, Feb 01, 2008 at 04:58:51PM +0100, Wolfram Sang wrote:

  Hi Wolfram,

we had the problem, that an embedded system ran out of memory after
parsing a huge XML-file. The problem was, that most of the memory was
only returned to the kernel after the process was terminated. When I
searched the xmlsoft-site, I found people having similar problems,
only a solution could not be provided (Example: )

Of course, there is NO memory leak and references to OS memory
management are correct. Still, there is a way (at least with glibc) to
enforce giving back the memory: malloc_trim

I am very sceptical if this functions is portable enough to be used
within libxml2.

 Actually portability wouldn't concern me that much, that could be tested
automatically, what makes it not acceptable is the amount of inpredictability
this would bring to the API. memory compation can be fairly expensive, and
calling it explicitely within one of libxml2 entry point could turn into
a serious problem. This is best left to the application programmer when
it actually needs it.

Still, I would suggest mentioning it in the docs
("Memory management"?), as this might point users into the right
direction. A paragraph could look like this:

  Now, that's a very good point. This wasn't raised that many time but
i can understand how it could be useful to some users, agreed !


Returning memory to the kernel

You may encounter that your process using libxml2 does not have a
reduced memory usage although you freed the tree. This is because
libxml2 allocates memory in a number of small chunks. When freeing one
of those chunks, the OS may decide that giving this little memory back
to the kernel will cause too much overhead and delay the operation. As
all chunks are this small, they get actually freed but not returned to
the kernel. On systems using glibc, there is a function call
"malloc_trim" from malloc.h which does this missing operation (note that
it is allowed to fail). Thus, after freeing your tree you may simply try
"malloc_trim(0);" to really get the memory back. If your OS does not
provide malloc_trim, try searching for a similar function.


  Great !
I am adding it to SVN, it should show up on the web site soon,
thanks a lot for the suggestion and the wording,


Red Hat Virtualization group
Daniel Veillard      | virtualization library
veillard redhat com  | libxml GNOME XML XSLT toolkit | Rpmfind RPM search engine

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]