Re: QA Team

[Andre Klapper <ak-47 gmx net>]

Adding numerous questions. 
I don't expect answers from anybody, just what comes to my mind.

On Fri, 2014-02-07 at 19:03 -0800, Sriram Ramkrishna wrote:
> The idea is to test everything wtih as much automation as possible
> while finding interaction bugs in the manual test.

Just FYI (not asking for making this a part here), in some (admittedly
very few) areas, basic checks via git pre-commit hooks could be done
even *before* merging a patch which will break things. 
For example the ever-recurring "the list of translated application
keywords must end with a semicolon!":
https://bugzilla.gnome.org/show_bug.cgi?id=723138

> * define how QA interacts with the release team - what tests would
> release team want to decide the quality of a release

GNOME is released on a time-based schedule, and I cannot remember that
we've ever postponed a release in the last seven years because it was
too buggy. (In hindsight, it could have been the case for one or two.)

I assume we talk about specific modules having issues, so we could
contact maintainers and tell them to fix X and create a new tarball.
That would require changing our current "Monday tarball, Wednesday
release" routine - having only two days is often already too tight.

> * UI automation tests will be done with dogtail

Out of curiosity, why dogtail and not e.g. LDTP? I assume there's more
expertise in our community and contributor base for dogtail?

Who will write and maintain the tests for project XYZ? If it's the QA
team it needs to be closely integrated with the development team of
project XYZ (similar to user documentation team) to keep things
up-to-date and know when things changed intentionally: Who is supposed
to update the test if the UI has changed and the test obviously breaks?

I'm asking because I've seen projects like Evolution getting LDTP tests
years ago, just to see them not getting updated and in the end ignored,
due to lack of (wo)manpower.

> * Document how to download an image, and run manual UI tests
> * Define manual automated tests for each of the components

"manual automated"? Is that "I manually start an automated test"? :)

Cheers,
andre
-- 
Andre Klapper  |  ak-47 gmx net
http://blogs.gnome.org/aklapper/