Re: [xml] libxml and threads

On Feb 23, 2005, at 11:38 AM, Daniel Veillard wrote:

On Wed, Feb 23, 2005 at 09:54:27AM +0100, Massimo Cafaro wrote:
Everything works fine, but I have found that many memory leaks are
associated with libxml2 APIs. The leaks are small, but they are so many
that this is a serious problem. Now, I strongly believe that it is
entirely my fault, but I need to understand exactly how to use libxml2
in threaded applications.

Leak: 0x0110f690 size=48 string 'MUTX'
Call stack: [thread 280c000]: | 0x0 | _pthread_body |
thread_starter | provider_thr | grelc_get_sql |
xmlSAXParseMemoryWithData | xmlCreateMemoryParserCtxt |
xmlNewParserCtxt | xmlInitParserCtxt | xmlDefaultSAXHandlerInit |
__xmlDefaultSAXHandler | xmlGetGlobalState | xmlNewGlobalState |
xmlInitializeGlobalState | xmlInitGlobals | xmlNewMutex | malloc |

The threads code is the only one which can legally do direct malloc()
without using xmlMalloc(), so it's among the memory use I won't detect
leaks in regression tests.

I do not understand this. What is exactly what you call threads code? The leaks originates in the xmlParseMemory() function, which is called in my application many times in many threads. I do not know exactly how do you check for leaks, I am using a standard tool on Mac OS X. May be it is worth saying here that the leaks are exactly the same on linux, as detected by valgrind.

It looks like a mutex object is never deallocated.

The code looks like

xmlDocPtr doc;

doc = xmlParseMemory (docname, strlen(docname) - 1);

Is this the way to write this kind of code? I was not able to find any
example of threaded code. I have found some APIs for dealing with
threads tough.

You must make sure you made the call to the library initialization
xmlInitParser() before any call to the library in the main thread,
and make sure you never call xmlParserCleanup() unless nothing will
ever use the library or library produced data.

Thank you for the suggestion. I will try this and let you know about.
mutex object and call related lock/unlock functions). The libxml2 I am
using is the latest one officially released on January. I am working on
Mac oS X 10.3.8

I don't use Mac oS X, I know Apple people now use libxml2 and they
should be able to track down that problem better than me. It is a
thread local storage problem and is usually platform specific.

Unfortunately, as i told you, the same leaks also appear on linux.

memory you're pointed out should be freed by xmlCleanupGlobals()
which is called by xmlCleanupParser() and to be released only once
in the whole application lifetime.

I will also try this.

I can't see how this specific call
can lead to " many memory leaks".
I.e. I looked at your problems but don't see how this could lead to
what you are describing.


Because in my app i have may threads calling many times libxml2 functions, especially the xmlParseMemory() function.

Thank you for your immediate reply, I really need to get rid of those leaks.
I will try immediately your suggestion, I will let you know what happens.




Massimo Cafaro, Ph.D. Center for Advanced Computational Technologies
Assistant Professor Voice +39 0832 297371
University of Lecce, Italy Fax +39 0832 297279
Faculty of Engineering E-mail massimo cafaro unile it
Via per Monteroni cafaro cacr caltech edu
73100 Lecce, Italy Web


Attachment: PGP.sig
Description: This is a digitally signed message part

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]