Re: [xml] after xpath search, xmlXPathFreeObject() can generate segfault

["Christopher J. Grayce" <cgrayce aleks com>]

Dear Daniel,

   Thanks very much for the response.  I always appreciate further insight
into how libxml2 works. 

   I think, however, I must have expressed myself badly in my first post.
In order to be sure what happened is quite clear to everyone, I have made some
responses to what you wrote.  I apologize if I am wasting bandwidth here
on issues already clear to everyone; I am thinking what is here may yet
be worthwhile to someone new to the library:

On Fri, Jan 09, 2004 at 04:33:33AM -0500, Daniel Veillard wrote:
> On Thu, Jan 08, 2004 at 11:21:38PM -0800, Christopher J. Grayce wrote:
> > Dear All,
> >
> >    I have a program that uses xmlXPathEval() to produce a list of nodes in
> > an xmlXPathObject.  Those nodes are then further processed, in the course
> > of which they are all deleted.  Finally, on exit, I call
>
> Rule 1: the objects returned from an XPath query are "live" objects
>         so if you delete them you delete them from the document being
>       queried.

I realize this.  In fact, I am counting on this.  The purpose of my XPath 
query is to find certain nodes in the tree in order to delete them.  

> > xmlXPathFreeObject() to clean up the xmlXPathObject -- but this generates
> > a segfault, because of the following code in xpath.c:
> >
> > --------------------------------------------------------------------------------
> > xmlXPathFreeNodeSet(xmlNodeSetPtr obj) {
> >     if (obj == NULL) return;
> >     if (obj->nodeTab != NULL) {
> >     int i;
> >
> >     /* @@ with_ns to check whether namespace nodes should be looked at @@ */
> >     for (i = 0;i < obj->nodeNr;i++)
> >         if ((obj->nodeTab[i] != NULL) &&
> >  -->        (obj->nodeTab[i]->type == XML_NAMESPACE_DECL))
> >             xmlXPathNodeSetFreeNs((xmlNsPtr) obj->nodeTab[i]);
> >     xmlFree(obj->nodeTab);
> >     }
> >     xmlFree(obj);
> > }
> > --------------------------------------------------------------------------------
>
>    This is an exception to Rule 1: due to difference in the implementation of
>      namespace nodes between libxml2 internal representation and the XPath
>      data model, the namespace nodes returned from XPath queries are not
>      coming from the document, but generated on the fly and destroyed
>      when the node set is destroyed.

This is quite interesting.  I wonder if xmlXPathEval() could keep a record of
such "fake" nodes, so that xmlXPathFreeNodeSet() knows exactly which node pointers
it can safely dereference?

> >    Right at the --> this code (from libxml2 2.6.4 CVS version of
> > 1/6/04) deferences a pointer which, since I have deleted the node, is
> > no longer valid.
>
>    Simply do not delete namespace nodes. They won't affect the tree anyway.

This is where I fear I gave the wrong impression.  I am not deleting 
namespace nodes.  The problem is not that I'm deleting nodes I don't want to
delete, or that the tree is being corrupted.  

I am deleting only nodes which I fully intend to delete, and the tree remains
completely consistent.  No problem there.  

The problem occurs when I clean up after the XPath query and tree modification.
The problem is a segfault in xmlXPathFreeObject() when it is asked to free
up the memory used in the XPath query.

The reason for the segfault is that xmlXPathFreeObject() assumes that the
pointers in the object which it is asked to free are either zero or still
valid.  

In other words, xmlXPathFreeObject() assumes that between the XPath query
which generated the object and when xmlXPathFreeObject() was called to free
the object, (1) there was no modification of the tree, or (2) any node
pointers which are no longer valid have been set to zero.

> >    My point in mentioning this is: I haven't seen it mentioned before that
> > if you free a node resulting from a call to xmlXPathEval(), you had better
> > set the pointer in the xmlXPathObject nodeTab[] vector to zero before
> > calling xmlXPathFreeObject().
>
>    No, it's an artefact due to the special allocation of namespace nodes.
> And anyway, what did you think removing namespace nodes meant ?

Again, I fear I gave the wrong impression.  What removing namespace nodes
means isn't what I'm interested in here.  The point is to tell newcomers
that *IF* you modify the tree after creating an xmlXPathObject object,
*THEN* you must set any affected pointers in the object to zero before 
you call xmlXPathFreeObject().

I didn't think this fact was completely obvious when you contemplate the 
API.  Hence my original message, which I hope will help anyone searching
the archive for information on mysterious segfaults after calling 
xmlXPathFreeObject().

> Daniel
>
> -- 
> Daniel Veillard      | Red Hat Network https://rhn.redhat.com/
> veillard redhat com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
> http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

Thanks again for your response, and of course for libxml2 itself.

Christopher