Re: [xml] libxml2-2.6.22 tests

[Kasimier Buchcik <K Buchcik 4commerce de>]

Hi,

On Fri, 2005-12-09 at 04:29 -0500, Daniel Veillard wrote:
> On Thu, Dec 08, 2005 at 03:44:14PM -0800, Manish Marathe wrote:
> > Hello,
> >
> > If I run libxml2-2.6.22 tests by two different ways
> >
> > 1 way:
> >
> > ./runtest
> > ./runsuite
> > ./testapi
> >
> >    a) 546 errors for Microsoft schema tests
> >    b) 27 errors for Sun schema tests
> >
> > 2) way:
> >
> > make tests / make check
> >
> >    a) 558 errors for Microsoft schema tests
> >    b) 38 failures and not errors for Sun schema tests
> >
> > Why is the number of errors for MS and SUN schema tests different in
> > both ways.
>
>   because it's not the same exact testing code.

'make tests' uses the Python test runner for the XML Schema test
suite; it's in xstc/xstc.py.
The difference is that the Python code will mark an _instance_ test as
failed if the _schema_ was already evaluated to be invalid. On the other
hand runsuite.c tries to validate the instance using XSI if the schema
was invalid.

> > Although the total number of tests ran for each of the schema
> > tests were same in both the cases, which are: 7230 for MS and 193 for
> > SUN
> >
> > Having that said which is the right way to execute tests?
>
>   depends what you're trying to do and your platform.

True.

However, I changed runsuite.c to eliminate the difference. You get
equal results from both sides now: runsuite.c revision 1.14.

Regards,

Kasimier