Re: Thoughts on GNOME Continuous improvements

[Lasse Schuirmann <lasse schuirmann gmail com>]

> > >  - Installed tests state transition; after each installed tests run,
> > > report on which tests
> > >    started failing

> One of the things I want is a build against a test or sandbox or whatever
> you want to call it.  Maybe sunday night football, I don't know. :-)  But it
> should allow anybody to commit to it and should auto revert commits that
> break the build.  That way we can have a branch doesn't have as much rules
> of acceptance than other branches and allows for some experimentation
> especially for younger people who don't really understand software engineer
> and the work required to do put something into main line.

TLDR: read key points down at "Concluding".

I think we should also take a look at where the rest of the world is
going. Looking at GitLab, Phabricator, GitHub and Atlassian
Stash/Bamboo there is a very clear trend: people are using git hooks
and continuous integration servers to check commits *before* they are
merged onto a branch that is considered stable. All those products
offer some kind of status API integrated to the commit/patch review
UI. This is awesome because this saves a massive amount of time for
maintainers and developers - anything that is not proper tested or
that fails tests can be automatically rejected/people can be advised
to correct it.

I really think we should try to achieve something similar for a higher
stability of our branches during the whole development cycle as well
as saving developer and maintainer time.

Looking at continuous integration services, those are usually
configured via a rather simple (usually yml) configuration file per
repository. This is awesome - have special needs, just throw them in
there. One can even use that to automatically enforce coding standards
and so on. If we want to make GNOME continuous fit for the future it
needs to be similarly easy to configure and extendable. (FWIW I have
no idea how tests are configured right now. This mystery deserves a
blog post and a properly placed and advertised wiki page.)

Actually there's even a few things more a continuous integration
server can help us with - a few extensions granted, just looking a bit
more into the future: why should a developer execute a certain set of
commands on his own every time for the same operation? There are
several of those, especially applying patches from BZ and releasing.
The chance of errors especially during the release process due to
simply forgetting to execute some tests can be reduced. The time
needed to push releases and patches can be reduced. A bot can do all
of this for us, given a solid continuous integration system exists
that is able to execute custom scripts. Such a system exists (under
the MIT license, IIRC) for GitHub: https://github.com/yegor256/rultor
/ rultor.com . I am using this for an own project and it's great
(apart from it being unstable and breaking from time to time, they
certainly lack some tests while we're at it :)). We can automatically
push releases to PyPI whenever we merge a Pull Request, we can upload
translations to our translation platform, releases are properly done
within minutes and there is no chance of forgetting to re execute the
tests in a sane environment, pulling the latest translations or
what-else-there is. Tedious processes can be reduced to one little
comment in the GitHub UI to summon the bot. Why not do something
similar? Maintaining a project doesn't have to be complicated.

Last but not least I would like to stress that I'd love to see GNOME
contributing to general and existing open source CI projects (jenkins
being only the most popular one) in this field instead of trying to
create/maintain our own service which is a hard job. I'm not into all
the technical details, but I think that contributing to an actively
developed software solution and extending it for our use cases may be
easier than actually build and maintain this all on ourselves and
would benefit a lot other people. GitLab CI is awesome, just saying.

Concluding:
-----------------

 * Commits should be checked before they get onto a stable branch,
ideally before they even get reviewed.
 * Ideally we'll want GNOME Continuous to check out patches from
bugzilla before a reviewer looks at them. (Placing a success/failure
comment on the patch.)
   * Producing an xdg app executable artifact would be awesome! Submit
a patch, continuous gives a link to the artifact, users and developers
can try out the patched version with no hassle at all.
 * Continuous must check stable branches too. To make sure master is
stable/protected, one might refuse manual pushes to master. Developers
would push to a "stage" branch, continuous might build that and after
this build succeeds continuous would fastforward master.
   * Alternative: rechecking and applying patches by a bot! No branch mess.
 * Configuration per repository (with sane defaults) is awesome and
allows projects to check against coding conventions and much more.
 * For the future: bots can help us applying patches from bugzilla and
doing releases. Continuous integration is a very important component
of this ability.
 * Is switching to a broader, actively developed CI system an option?

Greetings,

Lasse