Re: Performance Patches (was Re: [xml] too many mallocs?)

["Peter Jacobi" <pj walter-graphtek com>]

Hi Aleksey, All,

> IMHO, the performance bottleneck is memory allocations. 

Pretty much agree, but...

> Every call to
> malloc/free means global process lock/unlock and this really slows down
> everything. 

...for the exact amount of overhead you are at the mercy of your
library, as heap management in presence of multithreading can
be done with various amount of sophistication.

> However, in most cases you do not need to have every
> attribute, name, node structure, etc. allocated separatelly. [...] These points lead to the
> idea of doing "buffered" memory allocation in the xmlDoc: when memory is
> allocated to store a document node, node attribute, etc. libxml should get
> memory from *one* (per xmlDoc) growing memory buffer. 

This is the idea of the pluggable tree implementations mentioned in
the libxml3 thread. For read-only tree access one can do as you
described. With UTF8 input files (or after transcoding to UTF8) you even 
can mmap the file and let your pointers point directly into this memory.
The memory pool than only holds the Nodes, which gives very good 
locality (cache hits) when traversing the tree.

> This optimization mostly impacts multi-threaded application (most of the
> cases these days I guess). And it is worth to do it. However, this means a
> *lot* of changes in the code and I am not sure that it's the right time
> for this (probably this memory allocation should go to libxml3).

Totally agree. And keep in mind, that it seems that that each element 
name is malloced not once in the current implementation, but about four 
times. So, when we can fix this withing libxml2, 75% of the available 
gains can be earned without libxml3. (I'm not totally believing that we can 
reach one malloc per element name in libmxl2).

Regards,
Peter Jacobi