Emmanuele Bassi created an issue:
The current API reference publishing process works something like this
- a release artefact in the form of a tarball, containing generated HTML files
- or a tarball containing only the generated HTML files
- a cron job takes the tarball, extracts the HTML files, does some light text processing to change the cross-references from the developer's machine file system to the expected server file system
- the HTML files get copied into the appropriate location on a public facing web server
This process has recently started to fail pretty consistently because we stopped using Autotools for release tarballs, and every other build system is not self-hosting, which means it doesn't place generated files into the release artefacts. Using ad hoc tarballs of documentation is a manual process and a fairly intensive one at that, requiring SSH access to copy files into a known location—and configuring library-web to look for files in that location.
Never send a human to do a machine's job.
We have this CI thing that is more than capable of building the documentation of the GNOME platform's libraries. Sadly, we don't have a way to get the generated files copied into a well-known, public-facing location, because all our CI runners are stateless containers spun up whenever needed. At most, we can cache the data as an artefact, but of course we can't expect people to browse it properly. We could publish the API reference using GitLab pages, but pages blast away the existing data on deployment, and we need API references for different branches and different tags to coexist. There's an issue open for GitLab to do incremental deployment, but there's no milestone attached to it, and it's not really known if it's going to be available only for the Enterprise Edition clients or not.
A possible solution would be to have a dedicated runner VM that didn't use a stateless container; it would be a tagged runner, so that only specific projects with a job on their pipeline would be able to generate the API reference and copy it to a well-known location, watched by a web server. Nobody would have direct access to that server, and we could maintain an allow-list of projects that have access to it.
Requirements:
- a VM with a web server pointing at a well-known location—e.g.
/srv/pub/docs—and a GitLab CI runner with a well-known tag - a few projects to test a CI pipeline job for docs publishing—glib, gtk
- a DNS entry for testing—e.g. developer-beta.gnome.org
We would probably need some small script to rewrite the cross-references so that different projects can cross-link symbols.