[gnome-build-meta.wiki] Update OpenQA for GNOME developers



commit 076a13bdf5026703372c31c554f7e28ad4cf1fb2
Author: Sam Thursfield <sam afuera me uk>
Date:   Fri Aug 20 16:06:04 2021 +0000

    Update OpenQA for GNOME developers

 openqa/OpenQA-for-GNOME-developers.md | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
---
diff --git a/openqa/OpenQA-for-GNOME-developers.md b/openqa/OpenQA-for-GNOME-developers.md
index 05094db..38c9a7e 100644
--- a/openqa/OpenQA-for-GNOME-developers.md
+++ b/openqa/OpenQA-for-GNOME-developers.md
@@ -34,17 +34,33 @@ Tests can fail for several reasons. You will see the failure in the Web UI at ht
 
 First check the "Logs & Assets" tab. Most useful is `autoinst-log.txt`, containing the log of installing the 
ISO and running the testcases - this should log what went wrong. The `video.ogv` file can also be useful to 
see what went wrong.
 
-If you see "no candidate needle matched" or "`assert_screen` failure", this indicates a graphical change 
meaning the screenshot no longer matches. If the UI change is intentional then we will need to update the 
needle. See below.
+If you see "no candidate needle matched" or "`assert_screen` failure", this indicates a graphical change 
meaning the screenshot no longer matches. If the UI change is intentional then we will need to update the 
needle.
 
-If in doubt about a test failure, [open a gnome-build-meta 
issue](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues).
+The OpenQA Web UI lets you compare the actual screenshot with the expected ones. Select a needle in 
'Candidate needles and tags', then use the orange slider to see differences. The 'landscape' and 'eye' icons 
in the list switch between area-only or full diff. In the screenshot below, we have the expected image 
(installation completed) on the left, and the actual screenshot (installation still in progress) on the right.
 
+![image](uploads/e1b3415068ed01db4ce68aada5972b0f/image.png)
+
+If you think the test failure represents a bug, open an issue against the relevant component. Otherwise, or 
if in doubt, [open a gnome-build-meta issue](https://gitlab.gnome.org/GNOME/gnome-build-meta/-/issues).
 
 ## How to update a test
 
-OpenQA's [developer mode](http://open.qa/docs/#_developer_mode) allows you to
-update the needle from the web UI.
+Test scripts live in [gnome-build-meta](https://gitlab.gnome.org/GNOME/gnome-build-meta/) in the `openqa/` 
folder. If you need to modify the test itself, create a branch of gnome-build-meta as normal with your 
change. The test pipeline will run with your version of the tests, and once it passes we can merge this to 
master.
+
+Needles live in [openqa-needles](https://gitlab.gnome.org/GNOME/openqa-needles) folder, and the `master` 
branch is used by all tests. Multiple needles can have the same name, and OpenQA will pass a test if any one 
needle matches. If you need to modify a needle, the workflow is to create a new one, and the preferred way is 
to use OpenQA's [developer mode](http://open.qa/docs/#_developer_mode) as follows.
+
+ 1. Ensure you have 'operator' permissions on OpenQA - ask an admin or open an issue if you don't have this.
+ 2. Open the failed test in the Web UI.
+ 3. Click the 'Create new needle' button to open the developer mode page
+ 4. Use the 'Screenshot and Areas' section to create a new needle
+    * Usually you 'Take image from' the screenshot and 'Copy areas from' the most recent needle.
+    * Click and drag on the screenshot to create, move and resize match areas
+    * Click an area once to select it, enabling the "Selected area" controls
+    * Click on area again to change its [type](http://open.qa/docs/#_areas) - 'exclude' matches can be very 
useful when updating needles for UI changes.
+ 5. Now use 'Basics of Needle' to commit your change to 
[openqa-needles.git](https://gitlab.gnome.org/GNOME/openqa-needles):
+    * Write a short commit message describing why you made the change
+    * Click 'Save' to commit and push from the OpenQA webUI.
 
-**FIXME: do you need to be an admin to use 'developer mode'? Write more here.**
+Now rerun the test which hopefully passes.
 
 ## Adding more tests
 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]