Re: [gnome-desktop-testing] A thought about integrating Mago to desktop project source

Hi Tim:

Getting testing more tightly integrated would be awesome. There are at least a couple different ways to look at it, and it would be nice to try to have a test solution that helps with both:

1) I'm a developer and I want to test my changes before committing them.

2) I'm an integrator building from tarballs and would like to test how well everything plays together.

Other people also run manual tests (i.e., a human sits behind the machine and runs through a set of testing steps, recording the results).

Since you're already proposing a separate repository from GNOME git for tests, one thing that is also of interest to some people is Testopia. Testopia seems pretty nice for maintaining/organizing tests as well as recording results and provide result summaries. I believe it also has some way to integrate with Bugzilla. I'm curious if there might be a way to integrate with Testopia into the environment. For example:

1) The tests would be stored in Testopia.

2) The testing framework would pull the tests from Testopia, run them,
and optionally push test results back to Testopia.

3) The GNOME 'make test' targets could be set up to refer to the Testopia repository. Imagine also that more diligent projects might use Bugzilla to track patches and would require a pointer to a successful test run before the patch is committed.

4) Something else would provide the integrators with a way to get to the same tests (or perhaps various subsets of them - a smoke test, an exhaustive test, etc.).

Or, maybe just keep the tests in the bsr/git repository as they are now, but somehow try to find a way to integrate the test result reporting in Testopia.


Tim Miao wrote:
Hi there,

The GNOME desktop testing project/Mago is stepping forward, more and
more people are involving and contributing to this great project. Ara
did make a great presentation on this year GUADEC, and presentation
attracted much people there, people show their interests on this
automating testing framework.

This reminds me, could we integrate this automation testing framework to
the desktop project source code? To add a testing line in makefile, so
that each time when developer changes the code, builds the application,
the automation testing will be executed to make sure there will be no
serious regression introduced.

Mago/GNOME desktop testing has set up a bzr/git repository to store the
automation test cases for some application, and test cases are still in
developing. To introduce the automation testing to project source will
help both project owner to find out bugs and Mago/GNOME desktop testing
team to implement more automated test cases. Everyone will benefit from
this, I think.

I discussed this idea with Ara in irc channel, she also agreed with this
proposal, so I'd like to send this idea out for getting more feedbacks.
Any thoughts, suggestions and comments are always welcome. If GNOME
community could accept this, I think that would be a great thing for
both developers and QAs.

Thank you.
-Tim Miao
Sun Desktop QA Team

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]