Re: [xml] Possible bug with schema validation in combination with xsi:nil

[Thomas Gamper <icicle cg tuwien ac at>]

Am 28.02.2013 12:07, schrieb Daniel Veillard:
> On Fri, Feb 08, 2013 at 02:04:34PM +0100, Thomas Gamper wrote:
> > Am 08.02.2013 11:09, schrieb Thomas Gamper:
> > > Am 07.02.2013 17:59, schrieb Thomas Gamper:
> > > > Am 07.02.2013 17:56, schrieb Csaba Raduly:
> > > > > On Thu, Feb 7, 2013 at 4:47 PM, Thomas Gamper  wrote:
> > > > > > Hi!
> > > > > >
> > > > > > I ran into issues with libxml2 xml validation with schemas
> > > > > > as soon as i
> > > > > > started using the xsi:nil attribute. Attached you find a
> > > > > > reduced version of
> > > > > > the schema I am using (zenotrack-pilot.xsd + all the
> > > > > > dependencies it needs)
> > > > > > and two test cases:
> > > > > >
> > > > > > GetNextMessageResponse.xml which does not validate with with xmllint
> > > > > > (xmllint --schema zenotrack-pilot.xsd GetNextMessageResponse.xml)
> > > > > > GetMessagesResponse, which is extremely similar to the first
> > > > > > testcase, but
> > > > > > does validate with xmllint (xmllint --schema zenotrack-pilot.xsd
> > > > > > GetMessagesResponse.xml)
> > > > > >
> > > > > > I did try libxml2 2.7.7, 2.7.8 and 2.9.0. I did a debug
> > > > > > build of libxml2
> > > > > > 2.9.0 on Win64 and stepped with the debugger through the
> > > > > > code, but it is
> > > > > > simply to complex for me to understand what is actually
> > > > > > going on. All I saw
> > > > > > was that the nil attribute was recognized as such, but later
> > > > > > one validation
> > > > > > failed for reasons I do not understand.
> > > > > >
> > > > > > Probably I am not seeing the obvious, but still I would like
> > > > > > to ask for help
> > > > > > if there is actually something wrong with libxml2 in this case.
> > > > > It is almost certainly a bug in libxml2: I tried your files with a
> > > > > validator based on Xerces-C and both passed.
> > > > >
> > > > > Csaba
> > > > The validator integrated into Visual Studio seems to validate
> > > > both files fine too.
> > > >
> > > > Thanks,
> > > > TOM
> > > Ok, I found where the different behavior comes from:
> > > xmlschemas.c line 26072 where xmlRegExecPushString is called
> > > inside xmlSchemaValidatorPopElem, right when we are leaving the
> > > offending node (Message element, child of GetNextMessageResponse
> > > element)
> > >
> > >      libxml2.dll!xmlRegExecPushStringInternal(_xmlRegExecCtxt *
> > > exec=0x000000000cb6e200, const unsigned char *
> > > value=0x0000000000000000, void * data=0x0000000000000000, int
> > > compound=0)  Zeile 3786    C
> > >      libxml2.dll!xmlRegExecPushString(_xmlRegExecCtxt *
> > > exec=0x000000000cb6e200, const unsigned char *
> > > value=0x0000000000000000, void * data=0x0000000000000000)  Zeile
> > > 4133    C
> > >      libxml2.dll!xmlSchemaValidatorPopElem(_xmlSchemaValidCtxt *
> > > vctxt=0x000000000c973890)  Zeile 26072 + 0x13 Bytes    C
> > > >     libxml2.dll!xmlSchemaVDocWalk(_xmlSchemaValidCtxt *
> > > vctxt=0x000000000c973890)  Zeile 27977 + 0xd Bytes    C
> > >      libxml2.dll!xmlSchemaVStart(_xmlSchemaValidCtxt *
> > > vctxt=0x000000000c973890)  Zeile 28087 + 0xa Bytes    C
> > >      libxml2.dll!xmlSchemaValidateDoc(_xmlSchemaValidCtxt *
> > > ctxt=0x000000000c973890, _xmlDoc * doc=0x00000000080f9ba0)  Zeile
> > > 28166    C
> > >
> > >
> > > GetMessagesResponse
> > >
> > > xmlRegExecPushStringInternal is called, which in turn calls
> > > xmlRegCompactPushString in xmlregexp.c line 3787
> > > and returns 0.
> > >
> > > GetNextMessageResponse
> > >
> > > xmlRegExecPushStringInternal is called, which in turn does not
> > > call xmlRegCompactPushString,
> > > and returns -1.
> > >
> > > xmlschemas.c line 26073
> > > if ((ret<0) || ((ret==0) && (!INODE_NILLED(inode))))
> > >
> > > The check works with ret == 0, but not with ret = -1. This poses
> > > two questions:
> > > Is the check wrong?
> > > Or is the regexp matching doing something wrong?
> > >
> > > Thanks,
> > > TOM
> > So, attached you find a patch diffed against git master which fixes
> > the issue. There is no point doing a regexp validation of further
> > content if there actually *IS* no further content because the
> > element is nil. I also checked that validation of an element which
> > has xsi:nil="true" but contains child elements still fails as it
> > should.
>    Hi TOM,
>
>   thanks for checking the issue.
> I don't know XSD very well but that seems to make sense.
> However running the regression tests, "make check" now reports:
> ## Schemas regression tests
> Result for ./test/schemas/bug323510_0.xml on
> ./test/schemas/bug323510_1.xsd failed
> File ./test/schemas/bug323510_1.xsd generated an error
> Result for ./test/schemas/choice_4.xml on ./test/schemas/choice_0.xsd
> failed
> File ./test/schemas/choice_0.xsd generated an error
>
>    if I apply the patch.
> So my guess is that there is a side effect to your patch. Probably
> about the removale of the second test part where (ret == 0).
>
> So I rewrote it to minimize change to existing code paths and
> that smaller patch seems to work with your test case and pass
> make check. The specific testsuite run is also unmodified.
>
> So I pushed that:
>
> http://git.gnome.org/browse/libxml2/commit/?id=c100e69c4b4be13ba760eda45f72165e684c483f
>
>    can you double check it works for your full use case ?
>
>     thanks !
>
> Daniel
It's working fine. Thank you very much!

Cheers,
TOM