Re: [xslt] xsltproc memory consumption w/ large DTD / docbook

[Daniel Veillard <veillard redhat com>]

On Thu, Jan 30, 2003 at 11:36:53AM +0100, Michael Weiser wrote:
> On Wed, 29 Jan 2003, Daniel Veillard wrote:
> 
> > > Daniel: Can you give me a hint where to best start looking for freeing
> > > unneeded memory other than xsltLoadDocument?
> > superfluous DTD informations. get a debugger, put a breakpoint at the
> > place of my patch and see what's happening, I can't work on it
> > at the moment.
> Sorry for bothering you that much. Attached is a patch that does the trick
> for me. Memory consumption is down to 9MB and slowly grows up to about
> 15MB whilst processing that same autolayout.xsl that got OOM-killed at
> 900MB before. I'm just now re-transforming my whole website and it seems
> to work just fine for the other documents as well.

  Okay sounds fine then !

> I think the actual problem was that xmlGetIntSubset only returned the
> internal subset but not the external. Therefore I just grabbed the
> DTD-freeing code from xmlFreeDoc and put it into xsltLoadDocument. I'm
> sorry I couldn't integrate it less ugly but I know too little of
> libxml/libxslt coding to find the right place and way to do it.

  Right, I was suspecting something like this but hadn't had the
time to really look up. Totally removing the entities definitions
might not be a good idea, not that there might still be references
within the tree but I think one of the XPath base functions may still need
informations about entities ... so more scrutiny is needed before really
integrating the patch. Possibly by adding an option to xsltproc
to remove the DTD informations once loaded.

  thanks for the patch,

Daniel

-- 
Daniel Veillard      | Red Hat Network https://rhn.redhat.com/
veillard@redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/