Re: [xml] quite huge memory consumption

[Daniel Veillard <veillard redhat com>]

On Fri, Jul 24, 2009 at 01:43:22PM +0100, Grzegorz JaÅkiewicz wrote:
> Hi folks,
>
> I have a little XML file (13MB), containing 250 entries, scheduling
> information.
> Now, I am surprised to see, that xmlParseFile uses around 600MB to
> store information on that file.
>
> The way we parse it looks like that (pseudo code):
>
> xmlInitParser();
> doc = xmlParseFile(fn);
>
> context = xmlXPathNewContext(doc);
> s = xmlXPathEvalExpression(sTag, context);
> for(int j=0; j < s->nodesexyal->nodeNr; j++)
> {
>     if(NULL != s->nodesetval->nodeTab)
>     {
>       if(NULL!=(sNode = s->nodesetval->nodeTab[j]))
>        {
>
>           p = xmlGetProp(node, prop);
>              ...
>           xmlfree(p);
>
>           xmlDocSetRootElement(sDoc, sNode);

  And what did you do with all the previous root subtree ?
Well apparently you leaked it since I don't see any code destroying
it...

>           xy = xmlXPathEvalExpression(xyTag, context);
>
>            .........
>            r = xmlXPathEvalExpression(rTag, context);
>            ......
>            xmlXPathFreeObject(r);
>            xmlXPathFreeObject(xy);
>          }
>       }
> }
> xmlXPathFreeObject(s);
> xmlXPathFreeContext(context);
> xmlFreeDoc(sDoc);

 and here you free sdoc but not doc

> unlink(filename.c_str());
>
> xmlCleanupParser();
>
>
> so the memory consumption goes to 700 after xmlFileParse, and stays so
> high, even after xmlCleanupParser.
>
> any ideas ?

  Well I don't know what sdoc and snode are but obviously you're hiding
most of the informations on what you program actually does. Anyway this
list isn't the place to debug fractional pieces of code, it's trees 
and look at the function xmlCleanupParser()documentation it says
explicitely that it doesn't free any existing document.

Daniel

-- 
Daniel Veillard      | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
daniel veillard com  | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library  http://libvirt.org/