Re: [xml] Unable to create huge text nodes with libxml2 2.7.7

On Sun, Feb 27, 2011 at 02:15:19PM +0100, Michael Kwasigroch wrote:
Hi list,

I (actually a very important customer ;-) was bit by the 10MB size limit 
introduced in libxml2 2.7.3. Actually, we are using 2.7.7...

The code basically does the following (error handling omitted for clarity, 
actual code is more complex):

----- snip -----
xmlDocPtr        doc    = NULL;
xmlNodePtr       node   = NULL;
xmlTextWriterPtr writer = NULL;
char             huge[ 100*1024*1024 ] = { 'a' };
int              written;

doc = xmlNewDoc( (xmlChar *) "1.0" );
node = xmlNewDocNode( doc, NULL, (xmlChar *)"node", NULL );
xmlDocSetRootElement( doc, node );
writer = xmlNewTextWriterTree( doc, node, 0 );
xmlTextWriterStartElement( writer, BAD_CAST "huge" );
written = xmlTextWriterWriteBinHex( writer, huge, 0, sizeof( huge ) );
----- snip -----

With libxml2 V2.7.7 xmlTextWriterWriteBinHex() always returns -1 and error 
analysis reveals that the SAX2 parser hit the 10MB limit.

At first I didn't understood as xmlTextWriter is about generating a
serializer buffer output, but then I realized the problem comes from
the use of xmlNewTextWriterTree().

I was bypassing this by eliminating the size limit check in SAX2.c and 
recompiling libxml2 but I do not really like this "solution".

So the question is if there is a standard way of passing the 
XML_PARSE_HUGE option to the SAX2 parser used under the hood of the 
xmlTextWriter-family functions.

  IMHO xmlNewTextWriterTree is a hack, it forces a reparsing of the data
while it's not really needed, plus contrary to all other APIs invoking a
parser, there is no way of passing any options like you noticed.

I have done extensive research but was not able to find anything useful. I 
have checked but did not find a compile-time option to eliminate the 10MB 
size limit.

  My suggestion would be to directly create the parent node and its
children via the tree API instead of giving it to an intermediary


Daniel Veillard      | libxml Gnome XML XSLT toolkit
daniel veillard com  | Rpmfind RPM search engine | virtualization library

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]