Re: [xml] New libxml which is reentrant ?

[Daniel Veillard <veillard redhat com>]

On Tue, Jul 08, 2008 at 09:57:09PM +0200, Roland Mainz wrote:
> Daniel Veillard wrote:
> > On Tue, Jul 08, 2008 at 02:04:32AM +0200, Roland Mainz wrote:
> > > Are there any plans to make a new libxml version which is reentrant and
> >
> >   Define reentrant.
>
> The current problem we have is that if we have library "a" which uses
> libxml2 and then calls a function in library "b" which itself does some
> other stuff with libxml2 we get data corruption (the problem isn't
> limited to one library - we're seeing this in multiple libxml2 consumers
> in Solaris where libraries cannot be combined if both use libxml2 at the
> same time).

  Very strange. If that was true in general, that would mean your whole
Gnome interface system would be unusable, as it uses many libraries and 
much of them rely on libxml2 for various things. I think this libxml2
being used by many libraries is really a very common situation, not at
all Solaris specific
[root wei ~]# rpm -e libxml2 2>&1 | grep "is needed by" | wc -l
144
this includes a lot of GNOME libraries for example, including GConf
which in turn is used by any desktop app. Taking the example i'm 
most familiar with for virt-manager we use libxml2 heavilly both 
for libvirt virtualization level and for all the GNOME rendering.
If what you're suggesting was right it would pop up everywhere...

  By overgeneralizing, I'm afraid you discredit your case :-)

  "Does some stuff" ... is that a description of an enginnering problem ?
You may have an initialization problem. Possible that's described in a thread
before. Is that related ? How is that linked to being 'reentrant' ?
What kind of processing is going on with libxml2 ? What kind of corruption ?
If you can't analyze the problem based on facts, that means you don't
know what's going on and shooting in the dark. Sorry I don't have time for this.

> > > removes the use of global variables, e.g. replaces |xmlInitGlobals()|
> >
> >   no, though the use of global variable has been limited to strict minimum with
> > new APIs
> >
> > > with a function which returns a "handle" (similar to |_xmlGlobalState|)
> >
> >   no
> >
> > > which is per thread and/or consumer (similar to how PAM handles thread
> >
> >   global variables are already per thread
>
> Ok... but it seems this limits consumers in a way that they can't be
> nested or resources "moved" between threads (e.g. thread "a" creates a
> resource and then passes it completely over to thread "b" for further
> processing and disposal).

  I *really* don't understand what your problem is. 
If you use recent parsing APIs, basically the parsing would *not* rely
on any global variable state (except maybe predefined entities values
but they should have a predefined value which is not thread specific !)
Make 100% sure you are using the recent parsing functions of libxml2
those which don't rely on global variables for their processing.
Still I don't see how you would get data corruption. Maybe some of
your 'stuff' corrupts a parser or document dictionary. But I have
no idea.

The only way I can see data corruption coming in a multithread environment
while doing parsing is if multiple threads use the parser context
simultaneously. But you don't even specify what kind of processing you're doing
when you have the problem. I have no cristall ball, I'm just an engineer.

What I don't understand is how 

> > > support) and then passed explicitly to each consuming function ?
> >
> >   no
> >
> > In a nutshell, do not expect a libxml2 redesign.
>
> Do you know any other XML parsers which fall into the category of being
> reentrant (as described above) ?

  I have no idea. You really look desesperate, but without a proper 
engineering analysis of the problem I don't think I can help, sorry ...

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard redhat com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/