Re: [xslt] [Fwd: Re: slow key lookup in libxslt-1.1.22]

[Josef Urban <urban ktilinux ms mff cuni cz>]

Hi Bill,

On Mon, 24 Dec 2007, William M. Brack wrote:

> Hi Josef,
>
> Josef Urban wrote:
> > Hi,
> >
> > I am getting a huge slowdown after moving from libxslt-1.1.15 to
> > version 1.1.22 . I compiled both versions on the same machine,
> > against the
> > same libxml2 (2.6.30). The .xsl, .xml, profiles, and other files
> > needed
> > for re-running are at http://kti.ms.mff.cuni.cz/~urban/zfmodel1_ex/
> > .
> >
> > The timings are:
> >
> > time ../xsltproc.1.1.22 --load-trace --profile addabsrefs.xsl
> > zfmodel1.xml > /dev/null 2>profile.1.1.22
> >
> > real    1m40.778s
> > user    1m40.650s
> > sys     0m0.120s
> >
> > time ../xsltproc.1.1.15 --load-trace --profile addabsrefs.xsl
> > zfmodel1.xml > /dev/null 2>profile.1.1.15
> >
> > real    0m3.679s
> > user    0m3.560s
> > sys     0m0.110s
> >
> > About 30 times slower. The profiles (and experimenting) suggest a
> > bottleneck in a key lookup in the "abs" function, which basically
> > just
> > takes a name of a key $k (one letter code) and looks up an element
> > using
> > its number $nr: "key($k,$nr)".
> >
> > Any ideas what might be going on?
> >
> > Thanks a lot,
> > Josef Urban
>
> I spent quite a bit of time debugging this problem, including
> low-level (gcc / gprof) profiling of both libxslt and libxml2, and I
> think I have found and fixed it.

Excellent, thanks a lot!

> Since libxslt-1.1.15, there were many enhancements made to libxslt
> (and libxml2), mostly aiming to increase efficiency, but also fixing
> some problems.  The version which introduced this trouble was
> libxslt-1.1.17 (released in June 2006), when it was decided to only
> initialize "keys" for a document at the time they were first used.
> Unfortunately, there was a bug in the new code which resulted in the
> problem you encountered.  Instead of only initializing the keys
> once, with your stylesheet they were being "initialized" a very
> large number of times!

I see. I suspected some trouble with external documents, but just using 
"--load-trace" for debugging did not reveal that.

> My fixes are in SVN, and I would greatly appreciate it if you could
> try them and confirm that everything is now okay.  According to my
> tests, when compared to xsltproc from 1.1.15 the SVN version has a
> 16% improvement in "real" time, 5% improvement in "system", and 19%
> in "user" :-).

On the same machine with cvs snapshot from Dec 25:

time ../xsltproc.cvs071225 --load-trace --profile addabsrefs.xsl zfmodel1.xml > /dev/null 2>profile.cvs071225

real    0m3.347s
user    0m3.230s
sys     0m0.120s

The resulting document is the same. So yes, all looks very nice now.

> Thanks very much for your report.

Thanks very much for fixing this so quickly.

Best,
Josef