[gimp-web] devel-docs: new docs for developers/maintainers/admins about mirrors.

[Jehan <jehanp src gnome org>]

commit b90cfba5e4272cf91ebdac7279fff5eed4a0bc85
Author: Jehan <jehan girinstud io>
Date:   Tue Sep 28 23:44:57 2021 +0200

    devel-docs: new docs for developers/maintainers/admins about mirrors.
    
    This sums up how our current mirroring infrastructure works and how to
    update and maintain it. It also explains how to verify it regularly by
    doing security checks.

 devel-docs/mirror-howto.md | 127 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 127 insertions(+)
---
diff --git a/devel-docs/mirror-howto.md b/devel-docs/mirror-howto.md
new file mode 100644
index 00000000..37f847ac
--- /dev/null
+++ b/devel-docs/mirror-howto.md
@@ -0,0 +1,127 @@
+# Mirror administration
+
+## Architecture
+
+Mirrors are synced from the static download server `download.gimp.org`
+(a.k.a. `master.gnome.org`).
+Current infrastructure is a basic round-robin rotation managed through
+[Apache rewrite
+rules](https://gitlab.gnome.org/Infrastructure/puppet/-/blob/0df77787596314f41de55b270e016cfd99ce2a53/modules/httpd/files/sites.d/download.gimp.org.conf#L65-82)
+which will redirect `download.gimp.org/mirror/*` URLs to the same file
+at a random mirror from the list.
+
+The list of mirrors in rotation is found at: /etc/httpd/download.gimp.org.map
+
+Each file from this list should have associated rsync credentials
+(though some may not have any yet, if they were created from older
+process time):
+
+* rsync credentials are set in: /etc/rsyncd/secrets
+* the login part of the credential must also be copied in: /etc/rsyncd.conf
+
+The login must be in both files, otherwise syncing will not work.
+
+## New mirror procedure
+
+If someone requests to be an official mirror, the procedure they must
+follow is:
+
+* Create a mirror request on the [Infrastructure 
tracker](https://gitlab.gnome.org/Infrastructure/Infrastructure/issues/new?issuable_template=new-mirror)
+* Write in title and introduction that you want to be a mirror for GIMP in particular
+* Add `/cc @Jehan` in the request description or in a comment
+* Fill the relevant fields
+* If you want a specific name and URL to be shown in the mirror sponsor list, please write it explicitly in 
the report (otherwise we will choose whatever organization name and URL seems the most relevant from what you 
wrote down)
+* Once GNOME is done verifying your organization, they will send you rsync credentials for syncing your 
mirror
+* There is nothing to do in particular to appear on the Sponsors page which will be updated regularly 
through scripts. Yet it may take a few days or even weeks at times. So don’t worry if your organization name 
does not appear immediately!
+
+Note: a web URL will soon be available giving this procedure.
+
+Then a member of GNOME's Infrastructure team will take care of this
+request. Note that some members of GIMP may also have some rights, for
+instance @Jehan can take care of part of the actions:
+
+1. Verify the organization behind the request matches our expectations
+   in terms of being trustworthy. Some of the expectations are:
+   * they mirror other FOSS projects already (especially as official
+     mirrors)
+   * they are a well known organization which would gain from being a good
+     actor of the Free Software movement (unlikely to do anything shaddy
+     which would give a very bad reputation issue)
+   * if the mirror and claimed organization are different domain names,
+     verify they are the really linked (through `whois` or other means)
+2. Verify that the `https` URL has no major issue (our redirect happens
+   in https)
+3. If 1. and 2. are fine, generate an user and a password (e.g. with
+   `pwgen`) and add them to `/etc/rsyncd/secrets` in
+   `download.gimp.org`. This is done by directly editing the file.
+   Do not forget to leave a comment as well for reference (possibly a
+   Gitlab report link).
+   [*requires to be in proper admin group*]
+4. Once this is done, edit
+   
[/etc/rsyncd.conf](https://gitlab.gnome.org/Infrastructure/puppet/-/blob/master/modules/ftpadmin/files/rsyncd.conf)
+   by creating a Merge Request to
+   [Infrastructure/puppet](https://gitlab.gnome.org/Infrastructure/puppet)
+   (not editing it directly on the server, unlike 3.).
+5. Then once the MR is merged, send an email to the mirror admin (email
+   given in their report) asking them for their public GPG key.
+6. When they return their key, send the rsync credentials, encrypted with
+   this key, by email, and ask them to notify when the mirror is
+   properly set-up and synced. Wait for their answer.
+7. Once they notify you that the sync is complete, land the mirror on
+   the map file by editing `/etc/httpd/download.gimp.org.map` directly
+   on the server.
+   [*requires to be in proper admin group*]
+8. Close the report.
+9. on `gimp-web` repository, run:
+
+   ```sh
+   tools/downloads/update-mirrors.py --ssh-user <yourlogin>
+   ```
+   [*requires to have simple access to the server*]
+10. The script will update `tools/downloads/downloads.http.txt`
+   automatically and should tell you that
+   `content/downloads/mirrors.json` has to be updated too. Do so by
+   giving the public name of the mirror organization, link, location and
+   other data. In "more", also add the report link.
+   If other data needs to be updated, do so as well (for instance if
+   other mirrors changed).
+11. Commit all the changes and push them to `gimp-web`'s `testing`
+    branch.
+12. After a short time, make sure that testing's [sponsor 
page](https://testing.gimp.org/donating/sponsors.html)
+    is properly updated.
+
+Of course, the public website will be updated when you merge `testing`
+into `master` branch which does not have to happen immediately.
+
+## Mirror maintenance
+### Conf files must match
+
+Make sure that `auth users` in `[gimp]` section of `/etc/rsyncd.conf`
+are in `/etc/rsyncd/secrets`. If they are not, then they would not be
+able to sync anyway because they have no valid credentials, so you can
+remove them.
+
+Oppositely, users in the `secrets` file which are not in
+`/etc/rsyncd.conf` would not be able to fetch data. So if they never
+complained, it means they don't sync with us anymore.
+
+### Security checks
+
+From time to time, you could run:
+
+```
+tools/downloads/gimp-check-mirrors.py
+```
+
+By default, it would only check the last download tarball/installer/DMG.
+If you want to check a specific file, add it (or them) to the command
+line. If you set `--verify-checksum` option, then it will also check
+data integrity.
+
+## Future
+
+Some work is being done to move to a MirrorBits infrastructure which
+will bring even better download capability as it is able to choose the
+closer mirror to a downloader.
+
+It will also bring minimal download statistics.