RE: [xslt] RE: [xml] does xsltproc caches subexpressions

[Stefan Kost <ensonic hora-obscura de>]

Hi Kasimier,

I've been too keen to see it and just tried it. Please note that in
xsltproc the change need to be to call xmlXPathContextSetCache() and not
xmlXPathContextSetObjectCache(). It seems infact faster and most
noticable is the lower memory usage. And finally the generated documents
look good. So for now I'd love to see this activated by default.

Stefan

Am Dienstag, den 30.05.2006, 22:36 +0200 schrieb Buchcik, Kasimier:
> Hi,
> 
> > -----Original Message-----
> > From: Stefan Kost [mailto:ensonic hora-obscura de] 
> 
> [...]
> 
> > Am Montag, den 22.05.2006, 12:54 +0200 schrieb Buchcik, Kasimier:
> > > Hi,
> 
> [...]
> 
> > > The next bottleneck in the row is the template "indexterm" (mode =
> > > "reference")
> > > in "autoidx.xsl":
> 
> Next results: 
> 
> 
> 1) The expression "//index[&scope;][1]" was the bottleneck of the
>   "indexterm"-template.
> 
> We made the following enhancements to the XPath module:
> 2) Added an XPath object cache. This will avoid massive creation/freeing
>   of XPath objects. For the generation of the gst-docs 44 million
> objects
>   were created; after the cache was in use this dropped to 22 million
>   objects. If all slots of the cache are filled to maximum it will
> consume
>   about 40 KB additional memory.
> 
> 3) Enhanced xmlXPathNodeCollectAndTest(), which is the central function
>   for evaluation of steps.
> 
>  a) We eliminated massive recreation of xmlNodeSet structs; this was a
>   big bottleneck when traversing the descendant-or-self axis, since for
>   every traversed node a new xmlNodeSet was created; just count all
> nodes
>   of any type in the XInclude-processed input document of gst-docs and
>   you'll have the number of the structs created for every evaluation of
>   this axis.
> 
>  b) The following comes from the ChangeLog, since I don't want to
>   invent another explanation:
>     Optimized xmlXPathNodeCollectAndTest() and
>     xmlXPathNodeCollectAndTestNth() to evaluate a compound
>     traversal of 2 axes when we have a "//foo" expression.
>     This is done with a rewrite of the XPath AST in
>     xmlXPathRewriteDOSExpression(); I added an additional field
>     to xmlXPathStepOp for this (but the field's name should be
>     changed). The mechanism: the embracing descendant-or-self
>     axis traversal (also optimized to return only nodes which
>     can hold elements), will produce context nodes for the
>     inner traversal of the child axis. This way we avoid a full
>     node-collecting traversal of the descendant-or-self axis.
>     Some tests indicate that this can reduce execution time of
>     "//foo" to 50%. Together with the XPath object cache this
>     all significantly speeds up libxslt.
> 
> So the previous most-time-consuming templates were:
> 
> number       match       name            mode       Calls Tot 100us  Avg
> 
> 0            indexterm                     reference 2464 13039183
> 5291
> 1                         gentext.template          16047  2219472
> 138
> 2                        user.head.content             53  2216486
> 41820
> 3                                    chunk         191008  1984551
> 10
> 4                    *                    recursive-chunk-filename
>                                                     92686   799234
> 8
> 
> 
> Current result:
> 
> number       match       name            mode       Calls Tot 100us  Avg
> 
> 0            indexterm                     reference   2464 3425896
> 1390
> 1                                    chunk           191008 1609874
> 8
> 2                         gentext.template            16047 1261323
> 78
> 3                               dbhtml-dir           140277  710035
> 5
> 4                    *                    recursive-chunk-filename
>                                                       92686  703808
> 7
> 5                        user.head.content               53  600561
> 11331
> 
> 
> Stefan, If would be great if you could try the current CVS HEAD of
> Libxml2/Libxslt for the gst-docs generation. It would be interesting
> if this fixes the issue even for Ed Catmur and his sparse-memory
> machine.
> But you still need to customize xsltproc.c in order to activate the
> XPath
> object cache. It is disabled by default, so we need to make people
> aware that it's there and can be activated if things run slowly.
> Add a call to xmlXPathContextSetObjectCache() in xsltproc.c after
> the creation of the transformation context:
> 
> ctxt = xsltNewTransformContext(cur, doc);
> if (ctxt == NULL)
>     return;
> if (ctxt->xpathCtxt)
>     xmlXPathContextSetObjectCache(ctxt->xpathCtxt, 1, -1, 0);
> 
> 
> Regards,
> 
> Kasimier