GNOME.org
 

Re: Dogfooding discussion

Hi Richard,

On Wed, 2018-06-20 at 17:22 +0100, Richard Maw via Buildstream-list wrote:

For anyone unfamiliar with the term, dogfooding means to use your own product,

see https://en.wikipedia.org/wiki/Eating_your_own_dog_food.


We've let our team develop how they feel most comfortable.
This predictably has not resulted in using BuildStream to develop BuildStream,
since it's a new tool to us when we start
and then changing afterwards is extra effort so a reason to change is needed.
It's been decided that now we have a reason, and that is dogfooding.

The purpose of this E-Mail is to state our intent
and solicit feedback on how we propose to go about it.

It is not our intention to advocate everybody develop BuildStream this way,
for now we're only concerned with our team,
though we hope we'll be able to recommend it as a superior experience
after ironing out kinks in the workflow.

We're interested in feedback,
since the risk of dogfooding is optimising for our use-cases,
so if our approach isn't close enough to what other users are doing
we could make the experience worse.


Hmmm, ok so my thoughts in general are: Making use of BuildStream in
the daily activities of developing BuildStream is good, and I think the
appropriate amount of that will fall into the right place if we let it
happen at the right time.

By just building BuildStream with BuildStream, we cover a very small
surface of the BuildStream featureset; it really is the freedesktop-sdk
project (and GNOME project) which ensures we get fairly good usage
coverage of our features. If developers need a real project to test the
BuildStream feature they are working on, they *really* should be using
freedesktop-sdk or gnome-build-meta as a sample.

As you mention, this could cause us to optimize for our own use cases;
worse than this, we could invent our own problems by contorting our
workflow into something which requires BuildStream, causing friction in
other areas of our development process (or inventing unnecessary
features).

Some comments inline...


---

The key points of how we intend to implement our new workflow are:

1.  We use a container with BuildStream installed in it.

    This is for practical reasons of developers have a preferred OS
    which BuildStream may not be packaged for natively,
    and in the absence of continuous packaging effort for that OS
    the alternatives of holding back BuildStream's dependencies from updating
    or people on that OS not using the latest BuildStream aren't acceptable.

    In the absence of effort to build for other container runtimes
    we're going to be using docker, since we already build images
    and have the `bst-here` script to build on from.


I think we need to keep our install story on supported platforms
working well (which we also need to use it).

I know we have the `bst-here` script but I rather discourage it's use;
I think it is used on OSX seems to be mostly a workaround for not
having BuildStream work natively on OSX.

We of course use Docker on gitlab to ensure that we always have strong
control of the input of tests (and to diversify the platforms on which
we're testing).


2.  We develop BuildStream in a workspace.

    We currently work with a clone of buildstream.git,
    making changes there and `pip install`ing as appropriate.

    We would instead have a project that includes a buildstream.bst
    and any dependencies necessary, and use `bst workspace open`
    to have our clone of buildstream.git.

    This implies either automation of changes to buildstream.git
    causing updates to the project that contains buildstream.bst
    and us pulling changes from automation into our workspace,
    or that updates to buildstream.git will imply that we now
    are responsible for updating the project that contains buildstream.bst.


This feels like adding "build" and "deploy" to a cycle which is
currently just "edit" and "test" (using the `pip3 install --user -e .`
approach).

Doing a lot of work from within a `bst shell` open on a workspace
sounds a bit more likely...


3.  We use `bst shell` to test BuildStream.

    Instead of having our container include all the dependencies
    that are necessary to run buildstream.git's test suite,
    we instead include those by defining them in .bst files.

    We use `bst shell` for both running the test suite and test builds,
    using `--mount` to bring the test project and your local caches in.


But as I understand it, this is currently not possible (maybe it will
be possible with future kernel developments ?).

At least there needs to be some research towards whether this is even
possible: as I understand it Flatpak cannot be run in a Flatpak, and as
we use the same container technology, I expect that BuildStream cannot
run a build launched from inside a `bst shell`.


4.  Having a Merge Request accepted results in container updates.

    We currently have CI jobs to create Docker containers
    with the latest version of BuildStream.

    Users of the `bst-here` script are already doing this to some degree.

    We may want to have a wrapper script for `bst`
    that periodically checks if there's an update to the container
    and notifies so that the developer can opt to update their container.

    While we can know to wait for our own changes to be included in a container
    we won't so easily notice other people's changes if we're busy on a task,
    and it would be too disruptive for updates to be mandatory.


This sounds convenient for `bst-here` users in any case.


---

There's also a related argument that we should be publishing distro packages.

This could take the place of using a container, but not immediately
since the current container building approach involves installing from source.

There are benefits and drawbacks of using a container
which may be worth considering when packages exist as an alternative,
but we can defer a decision on that to some degree,
since we could incorporate packages into the container build
by having a CI job between running the tests and building the container
that publishes to a package repository,
which the container can add as a package source to install from.


I think I would prefer to have not ever gone into the territory of
recommending using BuildStream within a container; it may be practical
for people in some scenarios (like Docker in GitLab), but there should
not be a need for launching it from a container as a tool on your own
host.



---

Since we'll need a base system capable of hosting BuildStream to use `bst shell`
it's possible we could provide containers based on that instead of a distro.

We could incorporate dogfooding of the freedesktop-sdk or Baserock,
into our dogfooding story if we were to build our buildstream.bst on top.

I suspect since we may end up needing to do most of this to support
testing buildstream.git with `bst shell`,
it may be a relatively small amount of extra effort to use it for our container.


On a similar subject, I'd like to get coverage for `bst source-bundle`, 
it would be nice if we used the freedesktop-sdk project to assert that
it's actually useful for the purpose it was created for:
  o Run source bundle on the dependencies you need for a system with
    BuildStream
  o Build the cross built runtime for another arch, plus kernel
    and init script
  o Boot the cross built runtime in a VM, the init script launches
    the build script generated by `bst source-bundle`
  o Once that compiles successfully in it's vm...
  o Run the BuildStream test suite on that built in that emulator

I think we're going to get dogfooding along the way, and in the right
places; I just don't think we should rally our efforts under the banner
of dogfooding.

Cheers,
    -Tristan
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]