Re: [xml] xmlReadFile/xmlReadMemory - Performance or Concurrency problem

[Daniel Veillard <veillard redhat com>]

On Tue, Nov 04, 2008 at 10:14:48AM +0100, Martin Trappel wrote:
> Hi there.
>
> I could use a few wild guesses, because I've quite run out of them:
> * libxml2 2.6.27
> * Windows XPsp2
>
> I have a process that is running approx 30 threads where one of this  
> threads is doing some calculations and network communication with a  
> hardware device. This one is the only thread that spends a measureable  
> time doing anything, i.e. it takes about 10% cpu).
> So far, no libxml2 involved.
>
> Now, in an additional thread, I start up a libxml parser to parse a 4MB  
> xml file. (When tested in isolation parsing of this file takes approx  
> 200-300ms).
> In this process, the parsing (xmlReadFile, or xmlReadMemory call with  
> file read into memory) takes btw. 2 sec and 12 sec. That ain't the  
> problem and of course I expected it to take longer due to heavy load.
> The problem now is, that the xmlRead* call takes 99% cpu resources which  
> causes the other thread to slow down so much that it fails due to a  
> fixed timeout for msg processing we have.
>
> What is really interesting now is, that when I add some artificial  
> cpu-load before or after xmlReadFile (some dummy calculations in a loop  
> for 10 seconds) that takes up 99% cpu as well, but the msg processing in  
> the other thread ain't aborted.
>
> Could this be due to many heap-allocations from  
> xmlReadFile/xmlReadMemory? Some other process global resource that could  
> be the cause?
>
> any guesses welcome!

  No idea. the Windows memory allocator gave us serious problems in the
past in face of realloc() use. Thread concurency may be a problem too.

Daniel

-- 
Daniel Veillard      | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
daniel veillard com  | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library  http://libvirt.org/