Re: [orca-list] regression tests

[Joanmarie Diggs <jdiggs igalia com>]

Hi Mike.

As those are typically run only by me on a system with a given
configuration, no, they are not something you can use to verify
compatibility on previous systems.

--joanie

On 03/16/2016 03:53 PM, Mike Gorse wrote:
> Hi,
>
> In SUSE Linux Enterprise Desktop/Server, we have a tendancy to mix and
> match versions of various things (upgrading GNOME while using an older
> ESR version of Firefox or vice versa, etc), so I've been looking at the
> tests in the tests subdirectory, trying to figure out whether I can
> reasonably use them to look for incompatibilities. When I run the tests,
> I get a lot of failures, some of them seeming somewhat trivial, so I'm
> wondering whether something on my system is different from what the
> tests are expecting (an old version of Firefox, possibly), or if the
> expected results just need to be updated. I'll paste a sample below.
>
> Also, I should probably file a bug for this, but, if an exact string
> match fails, then we try to compile the expected result into a regular
> expression, without catching exceptions, and not all expected results
> can be compiled into regular expressions (the string might have a ( with
> no closing ), for instance), resulting in a cryptic traceback if the
> exact comparison fails.
>
> -- 
>
> Test 2 of 17 FAILED:
> /home/orca/test/harness/../keystrokes/firefox/aria_alert.py:2. Up Arrow
> DIFFERENCES FOUND:
> - BRAILLE LINE:  'show it.'
> + BRAILLE LINE:  '"block" to hide and show it.'
> -      VISIBLE:  'show it.', cursor=1
> +      VISIBLE:  '"block" to hide and show it.', cursor=1
> ?                 ++++++++++++++++++++
>
> - SPEECH OUTPUT: 'show it.'
> + SPEECH OUTPUT: '"block" to hide and show it. '
> [FAILURE WAS UNEXPECTED]
>
> Test 5 of 32 FAILED:
> /home/orca/test/harness/../keystrokes/firefox/aria_button_dojo.py:5. Tab
> to first Create button
> DIFFERENCES FOUND:
> - BRAILLE LINE:  'Create push button'
> + BRAILLE LINE:  'Create  push button'
> ?                        +
>
> -      VISIBLE:  'Create push button', cursor=1
> +      VISIBLE:  'Create  push button', cursor=1
> ?                        +
>
> - SPEECH OUTPUT: 'Create push button'
> + SPEECH OUTPUT: 'Create  push button'
> ?                        +
>
> [FAILURE WAS UNEXPECTED]
> _______________________________________________
> orca-list mailing list
> orca-list gnome org
> https://mail.gnome.org/mailman/listinfo/orca-list
> Orca wiki: https://wiki.gnome.org/Projects/Orca
> Orca documentation: https://help.gnome.org/users/orca/stable/
> GNOME Universal Access guide:
> https://help.gnome.org/users/gnome-help/stable/a11y.html
> Log bugs and feature requests at http://bugzilla.gnome.org