Re: QA Team

[Vincent Untz <vuntz gnome org>]

Hi,

(cc'ing another old-timer from the release team, Frédéric)

Le vendredi 07 février 2014, à 19:03 -0800, Sriram Ramkrishna a écrit :
> Hello Release Team!
>
> We are planning on spinning up a QA Team and we're looking for some
> feedback on how we can integrate with the release team.  QA will
> greatly help us improve the quality of our releases, find out
> regressions before the public discovers them, and provide feedback to
> maintainers and designers.
>
> For instance, we would like to define the components that you consider
> should be QA'd as part of the release.  Since you control what
> components go into GNOME it stands to reason that you would be
> interested in knowing the quality of each of the packages that is part
> of that release.
>
> Our plan right now is to split between automated tests and manual
> tests.  We have not really defined what that is, but it would be
> helpful to start with the list of components that you would be
> interested it.  It could just be a jhbuild list gnome-suites-core-3.12
> or something.  But it would be good to get the direction.
>
> The idea is to test everything wtih as much automation as possible
> while finding interaction bugs in the manual test.  Now that we have a
> continuously integration thanks to Colin, I think we have the
> infrastructure to make this a reality.
>
> Phase 1:
> * define the list of components that will be QA'd
> * define how QA interacts with the release team - what tests would
> release team want to decide the quality of a release
> * define interaction between release and engagement team (who are
> doing the release notes) in order to define the features, regressions,
> and hatever else we might be interested in.
>
> Phase 2:
> * define the tests for each of the components
>   * what will we test, and what won't we test
>   * possible call for volunteers to define manual tests
> * UI automation tests will be done with dogtail
> * UI interaction tests will be done with a team of contributors
> * Unit tests will conceived and implemented by module maintainers for libraries
>
> Phase 3:
> * Document how to write UI automation tests using dogtail
> * Document how to download an image, and run manual UI tests
> * Define manual automated tests for each of the components
>
> Phase 4:
> * Call for volunteers to manual interaction tests on specific images
> * Call for volunteers to write automated UI tests
> * Write automated tests and integrate them into the continuous build server
>
> Phase 5:
> Profit!
>
> This is sort of a rough idea of what I'm planning.
>
> Vincent - I would love to get feedback from the opensuse QA person.

I think Frédéric will actually be the right person to discuss this: he's
been thinking about testing in the last few months.

Cheers,

Vincent

-- 
Les gens heureux ne sont pas pressés.