Re: [xml] XML regression test cases...

[Daniel Veillard <veillard redhat com>]

On Tue, Oct 11, 2005 at 02:01:56PM -0700, Jain, Nilesh wrote:
> On Tue, 2005-10-11 at 15:15 -0400, Daniel Veillard wrote:
>
> >   I will try to take an example putting this in parallel. If you were testing
> > the Mozilla rendering on a set of complex HTML pages, would you consider 
> > failure to match an exact rendering at the pixel level for those pages
> > to be wrong. Obviously no! This is the same for libxml2 processing on some 
> > complex processin/transformations, it will usually give exactly the same
> > output on two runs in different environemnt, but failure to do so doesn't
> > mean it's broken.
>
> I understand that and agree with you but one thing which still bothers
> me is when I run these regression test from 2.20 against code base of
> 2.22 and see around 51 test errors. Theoretically I should not see
> errors if ABI behavior is constant and running in same environment. Am I
> correct or missing something?
>
> Why I am looking from that point of view is if I include these test
> cases into LSB runtime for conformance, and let say spec is based on
> 2.22 and distro is running 2.24/2.25.. test cases which is confirming
> the specs should not fail in same environment. 
>
> BTW the error messages I get when I run 2.20 against 2.22 code base is:
> ## XML regression tests
> File ./test/xhtml1 generated an error

  Please look at the difference between the ./test/xhtml1 output with 2.20
against what is expected from 2.22 
  This was discussed on this list during the month of August IIRC.

> ## SAX1 callbacks regression tests
> Got a difference for ./test/ent2
> File ./test/ent2 generated an error

  Same thing. Could you make a diff and check by yourselves ?

> Got a difference for ./test/ent7

  I saw those changes, and validated them, otherwise they would not have gone
in CVS. I take a user viewpoint when I make those decisions, if you think
they should not have gone in, please explain why :-)

> ## Schemas regression tests
> Error for ./test/schemas/any3_0.xml on ./test/schemas/any3_0.xsd failed
> Error for ./test/schemas/bug303566_1.xml
> on ./test/schemas/bug303566_1.xsd failed

  yes the XML Schemas implementation improved between those 2 versions.
Don't worry, even Microsoft is being fixing their XML Schemas implementation
as you can see on xmlschemas-dev w3 org mailing list !
And the complexity of the spec even warrant the garantee that you will never
see change in interpretation of the spec, there are parts nobody is 100% sure
about. This doesn't mean XSD support in libxml2 is not useful, nor that
peopel are not using it.
  
> I also see one error message while running 2.22 code base against 2.22
> code.
> ## Schemas regression tests
> Error for ./test/schemas/derivation-ok-extension_0.xml
> on ./test/schemas/derivation-ok-extension_0.xsd failed

  bug in the test program itself.

Daniel

-- 
Daniel Veillard      | Red Hat Desktop team http://redhat.com/
veillard redhat com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/