Re: New GnomeGoal proposal: InstalledTests

[Tristan Van Berkom <tvb gnome org>]

On Fri, Apr 26, 2013 at 9:46 PM, Matthias Clasen <matthias clasen gmail com> wrote:

On Thu, Apr 25, 2013 at 10:45 AM, Colin Walters <walters verbum org> wrote:

>
>> Do we have (makefile) infrastructure to allow GTests to run both
>> uninstalled (during make *check) and installed?
>
> Not at this time; that'd be nice, but I think the jhbuild model
> mostly obviates the need for uninstalled tests, because jhbuild by
> nature is a sandboxed environment independent from the underlying
> system.

You are not going to get me to buy eagerly into a new installed tests
scheme for glib if it means that I have to give up make check.

I just wanted to jump in and mention that, I'd really really like to see
better all around relocatability of modules.

Ideally, I'd like to see the following things:

  o Ability to run installed tests, encapsulated in the jhbuild prefix

  o Ability to use the same test cases that I use in-tree as installed

     tests, hopefully by simply installing my in-tree tests somewhere

     (perhaps they would run without some of the custom env vars

     that I would use in-tree, but still re-use all the same test suites

     when installed)

  o Overall relocated D-Bus and settings environment

This last one is a really big deal, there are some really annoying
things right now, for instance I can in *no way* test GTK+ apps

in a jhbuild shell and use the theme installed in the jhbuild prefix.
It seems natural that when typing 'jhbuild shell', any processes
that run should be accessing the settings persisted somewhere
in /opt/devel... instead I would have to slave my whole system
just to see what an app looks like with the theme installed in /opt/devel.

GTestDBus has brought us at least part of the way there when
it comes to D-Bus, but if we run installed tests we'll want to disable
the relocation of services done by GTestDBus (so that some apps
and services can interact with other services installed into the

build prefix, instead of running in complete isolation which is
more appropriate for 'make check'). Perhaps this can be done
by hardwiring some features directly into GTestDBus, of course
in this case we still would want to run a separate session bus for
the jhbuild environment, so that the whole relocated installed
test suite does not interfere with a real active session.

Some few tests, I realize will always depend on how the system
is setup, perhaps require a specific compositor, or even wayland,

but surely 90% or more of the stack is software that is not system
dependant. It would be interesting to have a separate class of installed
tests which *absolutely* cannot run in a relocated environment, but
those would really be a corner minimum case.

I also realize this is a lot of work, but frankly better relocatability is
something that benefits our workflow (I would be able to actually
run Evolution Mail in jhbuild without interfering with my mail account,
I would be able to see what GTK+ apps look like to those select
few who are actually running the bleeding edge of GNOME on a

device)... i.e. it would not only beneficial for installed tests, but
it would help with installed tests too.