[gnome-build-meta.wiki] Add OpenQA docs
- From: Sam Thursfield <sthursfield src gnome org>
- To: commits-list gnome org
- Cc:
- Subject: [gnome-build-meta.wiki] Add OpenQA docs
- Date: Mon, 9 Aug 2021 16:15:10 +0000 (UTC)
commit b0c329d7510c2b33e671b26beff76728ffbb293b
Author: Sam Thursfield <sam afuera me uk>
Date: Mon Aug 9 18:12:36 2021 +0200
Add OpenQA docs
Home.md | 4 +-
openqa/OpenQA-for-GNOME-developers.md | 70 +++++++++++++++++++++++++++++++++++
2 files changed, 73 insertions(+), 1 deletion(-)
---
diff --git a/Home.md b/Home.md
index d9a2599..c1aab00 100644
--- a/Home.md
+++ b/Home.md
@@ -3,6 +3,8 @@
# GNOME OS Bootable images
- Nightly images [os.gnome.org](https://os.gnome.org/)
+- Automated testing of the images:
+ - [Using OpenQA](openqa/OpenQA-for-GNOME-developers.md)
- [Documentation about bootable images for virtual machines](Bootable-images-in-virtual-machines)
- [Documentation about bootable image for Pinebook Pro](Pinebook-Pro-image)
- [How to build your own components and deploy on the image using ostree](deploy-components-ostree)#
@@ -43,4 +45,4 @@ In general, for libraries used by several modules, the removal process will last
- Create MR and be sure at least other 3 members of the release team approve (thumbs up) it
- Notify the maintainer about the removal plans
- Notify the community about the removal so they have time to port
-- When core apps have been ported / enough time has pass, merge the MR
\ No newline at end of file
+- When core apps have been ported / enough time has pass, merge the MR
diff --git a/openqa/OpenQA-for-GNOME-developers.md b/openqa/OpenQA-for-GNOME-developers.md
new file mode 100644
index 0000000..ca40ffc
--- /dev/null
+++ b/openqa/OpenQA-for-GNOME-developers.md
@@ -0,0 +1,70 @@
+# OpenQA for GNOME developers
+
+Using OpenQA we can install the nightly GNOME OS images in a virtual machine,
+and simulate user interactions to test that it boots to a working graphical
+desktop. We can then do more integration testing of GNOME as a whole via the
+Shell and the core applications.
+
+OpenQA has some useful online documentation. Be sure to look at the
+[starter guide](http://open.qa/docs/#gettingstarted),
+[tests developer guide](http://open.qa/docs/#writingtests) and
+[users guide](http://open.qa/docs/#usersguide).
+
+## Running the OpenQA tests
+
+The OpenQA tests run as part of Gitlab CI and can be started manually as follows.
+
+On an open merge request:
+
+ 1. Wait for 'track' and 'build-x86_64' jobs to complete
+ 1. Start 'iso-installer-x86_64' job.
+ 1. Start 'test-iso-installer-x86_64' job.
+ 1. Visit https://openqa.gnome.org/ to see progress (a link will be printed in Gitlab CI log)
+
+To run the tests against 'master' or a branch with no open merge request,
+first [start a pipeline](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/pipelines)
+and then follow the steps above.
+
+We would like to run the tests for every MR, which requires us to first [speed up the iso-installer
job](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues/396).
+
+
+## When tests fail
+
+The test failure will be shown in the Web UI at https://openqa.gnome.org/.
+
+If the machine fails to install or boot this is likely a regression somewhere
+in GNOME or Freedesktop SDK.
+
+If the test fails because GNOME's UI has changed, we might need to update the
+test in question.
+
+In either case, the first step is to
+[open a gnome-build-meta issue](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues).
+
+
+## How to update a test
+
+A well-written test is robust against minor changes, but major changes to GNOME's
+UI will result in a test failure at the `assert_screen` step. In this case the
+[needle](http://open.qa/docs/#_needles) needs updating for a UI change.
+
+OpenQA's [developer mode](http://open.qa/docs/#_developer_mode) allows you to
+update the needle from the web UI.
+
+**FIXME: do you need to be an admin to use 'developer mode'? Write more here.**
+
+## Adding more tests
+
+OpenQA can be used to test anything that
+[gnome-build-meta](https://gitlab.gnome.org/GNOME/gnome-build-meta/) builds.
+We must be able to maintain the tests with volunteer effort, so try to write
+tests only around parts of the UI that won't change frequently.
+
+The [how to write tests](http://open.qa/docs/#_how_to_write_tests)
+documentation is your starting point for adding a new test.
+
+Some distributions are already testing GNOME with OpenQA, and you may find you
+can adapt an existing test. Here are some we know of:
+
+ * OpenSuSE: [web UI](http://openqa.opensuse.org/)
+ * Fedora: [web UI](https://openqa.fedoraproject.org/),
[tests](https://pagure.io/fedora-qa/os-autoinst-distri-fedora/blob/master/f/tests)
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]