[gnome-build-meta.wiki] Add OpenQA docs



commit b0c329d7510c2b33e671b26beff76728ffbb293b
Author: Sam Thursfield <sam afuera me uk>
Date:   Mon Aug 9 18:12:36 2021 +0200

    Add OpenQA docs

 Home.md                               |  4 +-
 openqa/OpenQA-for-GNOME-developers.md | 70 +++++++++++++++++++++++++++++++++++
 2 files changed, 73 insertions(+), 1 deletion(-)
---
diff --git a/Home.md b/Home.md
index d9a2599..c1aab00 100644
--- a/Home.md
+++ b/Home.md
@@ -3,6 +3,8 @@
 
 # GNOME OS Bootable images
 - Nightly images [os.gnome.org](https://os.gnome.org/) 
+- Automated testing of the images:
+    - [Using OpenQA](openqa/OpenQA-for-GNOME-developers.md)
 - [Documentation about bootable images for virtual machines](Bootable-images-in-virtual-machines)
 - [Documentation about bootable image for Pinebook Pro](Pinebook-Pro-image)
 - [How to build your own components and deploy on the image using ostree](deploy-components-ostree)#
@@ -43,4 +45,4 @@ In general, for libraries used by several modules, the removal process will last
 - Create MR and be sure at least other 3 members of the release team approve (thumbs up) it
 - Notify the maintainer about the removal plans
 - Notify the community about the removal so they have time to port
-- When core apps have been ported / enough time has pass, merge the MR
\ No newline at end of file
+- When core apps have been ported / enough time has pass, merge the MR
diff --git a/openqa/OpenQA-for-GNOME-developers.md b/openqa/OpenQA-for-GNOME-developers.md
new file mode 100644
index 0000000..ca40ffc
--- /dev/null
+++ b/openqa/OpenQA-for-GNOME-developers.md
@@ -0,0 +1,70 @@
+# OpenQA for GNOME developers
+
+Using OpenQA we can install the nightly GNOME OS images in a virtual machine,
+and simulate user interactions to test that it boots to a working graphical
+desktop. We can then do more integration testing of GNOME as a whole via the
+Shell and the core applications.
+
+OpenQA has some useful online documentation. Be sure to look at the
+[starter guide](http://open.qa/docs/#gettingstarted),
+[tests developer guide](http://open.qa/docs/#writingtests) and
+[users guide](http://open.qa/docs/#usersguide).
+
+## Running the OpenQA tests
+
+The OpenQA tests run as part of Gitlab CI and can be started manually as follows.
+
+On an open merge request:
+
+  1. Wait for 'track' and 'build-x86_64' jobs to complete
+  1. Start 'iso-installer-x86_64' job.
+  1. Start 'test-iso-installer-x86_64' job.
+  1. Visit https://openqa.gnome.org/ to see progress (a link will be printed in Gitlab CI log)
+
+To run the tests against 'master' or a branch with no open merge request,
+first [start a pipeline](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/pipelines)
+and then follow the steps above.
+
+We would like to run the tests for every MR, which requires us to first [speed up the iso-installer 
job](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues/396).
+
+
+## When tests fail
+
+The test failure will be shown in the Web UI at https://openqa.gnome.org/.
+
+If the machine fails to install or boot this is likely a regression somewhere
+in GNOME or Freedesktop SDK.
+
+If the test fails because GNOME's UI has changed, we might need to update the
+test in question.
+
+In either case, the first step is to
+[open a gnome-build-meta issue](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues).
+
+
+## How to update a test
+
+A well-written test is robust against minor changes, but major changes to GNOME's
+UI will result in a test failure at the `assert_screen` step. In this case the
+[needle](http://open.qa/docs/#_needles) needs updating for a UI change.
+
+OpenQA's [developer mode](http://open.qa/docs/#_developer_mode) allows you to
+update the needle from the web UI.
+
+**FIXME: do you need to be an admin to use 'developer mode'? Write more here.**
+
+## Adding more tests
+
+OpenQA can be used to test anything that
+[gnome-build-meta](https://gitlab.gnome.org/GNOME/gnome-build-meta/) builds.
+We must be able to maintain the tests with volunteer effort, so try to write
+tests only around parts of the UI that won't change frequently.
+
+The [how to write tests](http://open.qa/docs/#_how_to_write_tests)
+documentation is your starting point for adding a new test.
+
+Some distributions are already testing GNOME with OpenQA, and you may find you
+can adapt an existing test. Here are some we know of:
+
+  * OpenSuSE: [web UI](http://openqa.opensuse.org/)
+  * Fedora: [web UI](https://openqa.fedoraproject.org/), 
[tests](https://pagure.io/fedora-qa/os-autoinst-distri-fedora/blob/master/f/tests)


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]