Re: New GnomeGoal proposal: InstalledTests

[Colin Walters <walters verbum org>]

On Thu, 2013-04-25 at 16:41 +0100, Simon McVittie wrote:

> The currently-defined Restrictions in autopkgtest are rw-build-tree,
> breaks-testbed, needs-root and build-needed. The two build ones are not
> relevant for installed tests. GNOME could use needs-root, breaks-testbed
> (if it breaks the system) and breaks-home (if it trashes $HOME), perhaps?

For needs-root I am thinking now that the best way for these to work is
to quite simply install systemd units, sysv init scripts, upstart jobs
or whatever.  Disabled by default, obviously.

This model would be particularly good for systemd units and upstart jobs
because they already have expressive ways of hooking into the OS boot
order, which a lot of these needs-root tests are going to want.

We'd just need a naming scheme or something so that a tool could turn
them on programatically.  How about installed-test-$PKGNAME-$TESTNAME ?


As for breaks-home, let's add that on later; it's easily supported in
the gnome-ostree test framework, but obviously it's not something you'd
want in jhbuild =)    Although - with fast user switching, this could
actually work; jhbuild should probably support that.


Otherwise these are great, constructive comments, keep them coming!
Thanks all who replied so far.

I've adapted the code to use .desktop file like files (currently
called .testmeta), and updated both the wiki page:
https://live.gnome.org/GnomeGoals/InstalledTests#preview
and the code:
https://git.gnome.org/browse/gnome-desktop-testing/commit/?id=ed967e21209930356814af187b819a95258478d0
https://git.gnome.org/browse/gjs/commit/?h=wip/installed-tests&id=f7e4b445f085be31f9f409d0a968398b04f27859

Planning to push this live in the next day or so and we'll see how it
works out for gjs.  If any project for whom gnome-ostree is currently
doing continuous integration wants to add installed tests, please ping
me.