Re: [xml] libxml and threads

[Massimo Cafaro <massimo cafaro unile it>]

On Feb 23, 2005, at 11:38 AM, Daniel Veillard wrote:

> On Wed, Feb 23, 2005 at 09:54:27AM +0100, Massimo Cafaro wrote:
> > Everything works fine, but I have found that many memory leaks are 
> > associated with libxml2 APIs. The leaks are small, but they are so many 
> > that this is a serious problem. Now, I strongly believe that it is 
> > entirely my fault, but I need to understand exactly how to use libxml2 
> > in threaded applications. 
>
> [...]
> > Leak: 0x0110f690  size=48       string 'MUTX' 
> > Call stack: [thread 280c000]: | 0x0 | _pthread_body | 
> > thread_starter | provider_thr | grelc_get_sql | 
> > xmlSAXParseMemoryWithData | xmlCreateMemoryParserCtxt | 
> > xmlNewParserCtxt | xmlInitParserCtxt | xmlDefaultSAXHandlerInit | 
> > __xmlDefaultSAXHandler | xmlGetGlobalState | xmlNewGlobalState | 
> > xmlInitializeGlobalState | xmlInitGlobals | xmlNewMutex | malloc | 
> > malloc_zone_malloc  
>
> The threads code is the only one which can legally do direct malloc()
> without using xmlMalloc(), so it's among the memory use I won't detect
> leaks in regression tests.

I do not understand this. What is exactly what you call threads code? The leaks originates in the xmlParseMemory() function, which is called in my application many times in many threads. I do not know exactly how do you check for leaks, I am using a standard tool on Mac OS X. May be it is worth saying here that the leaks are exactly the same on linux, as detected by valgrind.


> > It looks like a mutex object is never deallocated. 
> >
> > The code looks like 
> >
> > xmlDocPtr doc; 
> > ... 
> >
> > doc = xmlParseMemory (docname, strlen(docname) - 1); 
> > ... 
> > xmlCleanupParser(); 
> >
> >
> > Is this the way to write this kind of code? I was not able to find any 
> > example of threaded code. I have found some APIs for dealing with 
> > threads tough. 
>
> You must make sure you made the call to the library initialization
> xmlInitParser() before any call to the library in the main thread,
> and make sure you never call xmlParserCleanup() unless nothing will
> ever use the library or library produced data.


Thank you for the suggestion. I will try this and let you know about.
> > mutex object and call related lock/unlock functions). The libxml2 I am 
> > using is the latest one officially released on January. I am working on 
> > Mac oS X 10.3.8 
>
> I don't use Mac oS X, I know Apple people now use libxml2 and they
> should be able to track down that problem better than me. It is a
> thread local storage problem and is usually platform specific. 

Unfortunately, as i told you, the same leaks also appear on linux.

> The
> memory you're pointed out should be freed by xmlCleanupGlobals() 
> which is called by xmlCleanupParser() and to be released only once
> in the whole application lifetime.

I will also try this.


> I can't see how this specific call
> can lead to " many memory leaks".
> I.e. I looked at your problems but don't see how this could lead to
> what you are describing.
>
> Daniel


Because in my app i have may threads calling many times libxml2 functions, especially the xmlParseMemory() function.

Thank you for your immediate reply, I really need to get rid of those leaks.
I will try immediately your suggestion, I will let you know what happens.

Cheers,
Massimo

-- 

*************************************************************************************

Massimo Cafaro, Ph.D.        Center for Advanced Computational Technologies
Assistant Professor               Voice  +39 0832 297371    
University of Lecce, Italy      Fax    +39 0832 297279     
Faculty of Engineering         E-mail massimo cafaro unile it            
Via per Monteroni                              cafaro cacr caltech edu
73100 Lecce, Italy                Web     http://sara.unile.it/~cafaro

*************************************************************************************

Attachment: PGP.sig
Description: This is a digitally signed message part