Re: dogtail-devel Proposal: splitting out the app wrappers into a separate tarball/repo, and how to organize dogtail-tests

[Zack Cerza <zcerza redhat com>]

David Malcolm wrote:
> [cross-posting to build-brigade-list as I'd like feedback on how best to
> structure this with jhbuild etc: see near the end of this post]
>   
[keeping this short and sweet because thunderbird blew up on my last 
attempt to reply. yay for kwality!]
> As Dogtail development moves on, I think we're going to want to
> stabilize the core APIs, and eventually touch these as little as
> possible.  
>
> However, one problem with this is the dogtail/apps/wrappers
> subdirectory.  These are full of little workarounds for dealing with
> specific apps (capturing these workarounds and distro-specific
> differences in one place, rather than forcing them to be cut and pasted
> throughout the tests) and I suspect we're going to want to continually
> make changes to this part of the code, even when the rest of the
> codebase is done.
>   
While the app wrappers can be a good idea in some situations, I've 
always thought they belonged in a separate place (module or otherwise) 
from the core of dogtail.

> So I think we should move this out of the main dogtail module and into
> somewhere else.  Doing so will break much of the existing example code,
> unfortunately.
>   
If we're moving the wrappers, we're going to need to move the examples 
that use them.
> What do we have here?  (lengthy, so skip this list to get to the
> proposal)
> - dogtail/apps/categories.py:  my feeling is that this was an
> experiment, and can be moved elsewhere or got rid of
> - dogtail/apps/wrappers/epiphany.py: short and somewhat hackish
> - dogtail/apps/wrappers/evolution.py: quite a lot here, useful
> - dogtail/apps/wrappers/gcalctool.py: not particularly useful
> - dogtail/apps/wrappers/gnomepanel.py: appears to contain some
> Fedora-specific code
> - dogtail/apps/wrappers/kicker.py: just a stub
> - dogtail/apps/wrappers/konqueror.py: just a stub
> - dogtail/apps/wrappers/mozilla.py: just a stub
> - dogtail/apps/wrappers/nautilus.py: short and somewhat hackish
> - dogtail/apps/wrappers/yelp.py: short and somewhat hackish
> - examples/abiword-test.py: never finished
> - examples/crack.py: never finished
> - examples/evolution-test-composing-html.py: never finished, but would
> make a good test of the evolution wrapper.
> - examples/evolution-test-configuring-exchange.py: works, but assumes
> evolution is already configured and running.  Uses evolution wrapper.
> - examples/evolution-test-configuring-imap-smtp.py: works, but assumes
> evolution is already configured and running.  Uses evolution wrapper.
> - examples/evolution-test-first-time-wizard.py: works IIRC; assumes
> evolution is running, and may blow away your configuration.  Uses
> evolution wrapper.
> - examples/evolution-test-sending-email.py: works.  Uses evolution
> wrapper.
> - examples/evolution-test-survives-email-CAN-2005-0806.py: works IIRC.
> Uses evolution wrapper.
> - examples/evolution-test-switching-components.py: works, but doesn't
> test anything (infinite loop).  Uses evolution wrapper.
> - examples/filechooser-stress-test.py: works; another infinite loop test
> that should be turned into a resource usage test.  Doesn't use wrappers.
> - examples/firefox-test-browsing-local-html-file.py: not finished; would
> use mozilla wrapper.
> - examples/gcalctool-test-fibonacci.py: silly demo, using gcalctool
> wrapper
> - examples/gedit-test-utf8-procedural-api.py: doesn't use wrappers.  
> - examples/gedit-test-utf8-tree-api.py: doesn't use wrappers
> - examples/gnome-panel-test-starting-every-app.py: silly non-test; uses
> gnomepanel wrapper
> - examples/google-search.py: silly non-test; uses epiphany wrapper
> - examples/i18n-test.py: actual example of using the i18n API, rather
> than a test case; doesn't use app wrappers.  Should stay in examples
> subdirectory
> - examples/nautilus-test-icon-view-collage.py: silly non-test; uses
> nautilus wrapper
> - examples/no-help-at-all.py: silly non-test that shows off the tree
> API; doesn't use app wrappers.  Should stay in examples subdirectory
> - examples/rhythmbox-test.py: unfinished stub
> - examples/test-events.py: simple example of using atspi event API, for
> debugging purposes.  Should stay in examples subdirectory
>
>
> Where should it go?  I think it would be useful to have a common
> location where Dogtail tests can live (although e.g. Frysk is storing
> its tests within its main source tree).  There's already a dogtail-tests
> module in GNOME CVS.  So I propose moving the bulk of the examples above
> to that module, with some kind of subdirectory structure.  Note that the
> current tests in dogtail-tests all use the wrappers, and so these may
> need fixing up too (there aren't many so this should be easy).
>
> With that in mind, I think it makes sense to add the app wrappers
> somewhere inside dogtail-tests, possibly with some Python packaging. I'm
> not yet sure how to organize this.  We need something that allows for:
> - jhbuild to be able to grab the wrappers and a collection of tests, and
> run them
> - ability to grab the wrappers and make them available to out-of-tree
> tests
> - easy use of the wrappers when developing tests.
>
> Possible approaches:  (i) have a per-application directory of tests (as
> we do now) and store the wrapper for each app within each directory.
> (ii) gather all the wrappers into some Python package that can be
> installed as a package by a distribution, or rolled up as a
> nicely-versioned tarball, be a single module within jhbuild etc
>   
My initial feeling is that we should go with (ii) and call the package 
something like dogtail_app_wrappers, but possibly shorter than that. I 
also don't want to give the impression that anyone must or must not use 
the wrappers. Keeping them separate from the actual tests might help there.

> We probably should store desktop-wide integration tests in another
> subdirectory of dogtail-tests too.
>   
Agreed. We could just call it 'integration'. Maybe capitalize the 'I' to 
make it stand out from the apps, also ;)

Cheers,
Zack