Re: [xml] XML regression test cases...

On Tue, Oct 11, 2005 at 02:01:56PM -0700, Jain, Nilesh wrote:
On Tue, 2005-10-11 at 15:15 -0400, Daniel Veillard wrote:

  I will try to take an example putting this in parallel. If you were testing
the Mozilla rendering on a set of complex HTML pages, would you consider 
failure to match an exact rendering at the pixel level for those pages
to be wrong. Obviously no! This is the same for libxml2 processing on some 
complex processin/transformations, it will usually give exactly the same
output on two runs in different environemnt, but failure to do so doesn't
mean it's broken.

I understand that and agree with you but one thing which still bothers
me is when I run these regression test from 2.20 against code base of
2.22 and see around 51 test errors. Theoretically I should not see
errors if ABI behavior is constant and running in same environment. Am I
correct or missing something?

Why I am looking from that point of view is if I include these test
cases into LSB runtime for conformance, and let say spec is based on
2.22 and distro is running 2.24/2.25.. test cases which is confirming
the specs should not fail in same environment. 

BTW the error messages I get when I run 2.20 against 2.22 code base is:
## XML regression tests
File ./test/xhtml1 generated an error

  Please look at the difference between the ./test/xhtml1 output with 2.20
against what is expected from 2.22 
  This was discussed on this list during the month of August IIRC.

## SAX1 callbacks regression tests
Got a difference for ./test/ent2
File ./test/ent2 generated an error

  Same thing. Could you make a diff and check by yourselves ?

Got a difference for ./test/ent7

  I saw those changes, and validated them, otherwise they would not have gone
in CVS. I take a user viewpoint when I make those decisions, if you think
they should not have gone in, please explain why :-)

## Schemas regression tests
Error for ./test/schemas/any3_0.xml on ./test/schemas/any3_0.xsd failed
Error for ./test/schemas/bug303566_1.xml
on ./test/schemas/bug303566_1.xsd failed

  yes the XML Schemas implementation improved between those 2 versions.
Don't worry, even Microsoft is being fixing their XML Schemas implementation
as you can see on xmlschemas-dev w3 org mailing list !
And the complexity of the spec even warrant the garantee that you will never
see change in interpretation of the spec, there are parts nobody is 100% sure
about. This doesn't mean XSD support in libxml2 is not useful, nor that
peopel are not using it.
I also see one error message while running 2.22 code base against 2.22
## Schemas regression tests
Error for ./test/schemas/derivation-ok-extension_0.xml
on ./test/schemas/derivation-ok-extension_0.xsd failed

  bug in the test program itself.


Daniel Veillard      | Red Hat Desktop team
veillard redhat com  | libxml GNOME XML XSLT toolkit | Rpmfind RPM search engine

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]