[BuildStream] Proposal: Adding support for CI on WSL

[Jonathan Maw <jonathan maw codethink co uk>]

Hi, I've been looking at how to make sure gitlab's CI covers buildstream 
being
run from WSL on a Windows system.

So far, I've managed to get a basic prototype working
(https://gitlab.com/BuildStream/buildstream/-/jobs/143383993) using a
pre-configured laptop provided by Phillip Smyth.

The explicit goals is to have a `gitlab-runner` instance running in
a Windows 10 system on WSL that runs pre-merge tests of buildstream,
so that we can ensure that merges do not break things for WSL users.

To fully provide the coverage, I think we will need:

1. A way of reliably producing a windows image that can run the tests in 
WSL.
   My understanding is that the only requirement is that we have a
   gitlab-runner serving buildstream's CI, as such, a single windows VM
   running somewhere secure will be sufficient.

   To ensure that this can be done, I will provide instructions for
   configuring a Windows 10 system from scratch to run gitlab-runner.

   Ideally, as much of the instructions will be shell commands that can
   be pasted into a terminal.

2. To fix the tests that are currently failing.

   There are a small number of tests that fail when moving over to WSL:

   a. The Plugin Source tests fail because the given path is too long to 
bind a unix socket.
      I have identified the problem, and an unmerged fix can be seen at
      
https://gitlab.com/BuildStream/buildstream/commit/21908aac7203ac96a4fb3c6acf350214e56d5e56

   b. The artifact cache expiry tests
      tests/artifactcache/expiry.py::test_never_delete_required and
      tests/artifactcache/expiry.py::test_never_delete_required_track 
fail.

      After some investigation, this appears to be because running out 
of
      artifact cache size doesn't abort the build.

   c. tests/sandbox/mounting/mount_simple.py::test_bind_mount fails when
      calling `/bin/mount --make-rslave /tmp/tmp08i96tr1`

      This test should be skipped.

3. To extend the tests so that they can use buildgrid to run builds.
   Jim MacArthur has been working on this, and my work towards getting 
support
   for CI on WSL will involve supporting him in continuing that work or 
picking
   it up myself depending on the priorities at the time.

   This is needed because buildstream on WSL doesn't currently support
   sandboxed builds, because Windows lacks a compatible FUSE 
implementation.


--
Jonathan Maw, Software Engineer, Codethink Ltd.
Codethink privacy policy: https://www.codethink.co.uk/privacy.html