BuildStream BoF at GUADEC: Monday session

From: Agustín Benito Bethencourt <agustin benito codethink co uk>
To: buildstream-list gnome org
Subject: BuildStream BoF at GUADEC: Monday session
Date: Tue, 10 Jul 2018 10:58:34 +0200

Hi,

this are the minutes taken during the Monday BuildStream BoF at GUADEC.

##################3

# BoF Session notes

* Improvements in Code Review governance. - @tristanvb print("B")
* Gitlab related topics
* Notifications through mail - @toscalix
* Evaluation of the capabilities to integrate Gitlab and mailing lists
to send merge requests and diffs to the mailing list. - @toscalix
* Bootable images (complete systems) - @jjardon
* Documentation - @toscalix
* BuildStream release
* Schedule - @toscalix
* Goals - @tristanvb @jjardon and @toscalix
* Key features - @tristanvb
* Release checklist - @toscalix
* Freedesktop SDK - Javier and Adam
* Building Flatpak apps with BuildStream- Valentin
* SourceCache - Richard Maw
* Dogfooding BuildStream - Richard Maw
* Architectural overview - Tristan

## Monday session

* Release team experience with BuildStream - Michael Catanzaro / Javier
Jardón
* VCS reduction (BoF) - Richard Maw
* 1.2 blockers - Tristan
* Startup performance - Emmet
* BuildStream on Windows and OS X - Richard Maw

* Moved to another session
* Merging CAS

### GNOME Release Team experience with BuildStream

* Pretty good replacement for JHBuild for release management purposes.
Smoothest release.
* Not smooth enough to replace JHBuild for developer workflow.
* Initialization of bst should be as fast as possible
* Issue: https://gitlab.com/BuildStream/buildstream/issues/466
* Awkward to switch between shells. Have to build dependent element,
switch shell, build again.
* Issue: https://gitlab.com/BuildStream/buildstream/issues/432
* Switching shells is slow.
* Integration commands makes it slow. Not needed most of the time.
* Need domains for integration commands to know when it's
necessary. Build integration or runtime integration has different
requirements. If we split it up we could save ~80% (tristanvb).
* Related issue: https://gitlab.com/BuildStream/buildstream/
issues/295
* Issue for integration commands: https://gitlab.com/BuildStream/
buildstream/issues/464
* No limit to size of cache directory. Fills up 1TB quickly. Needs to
be a low limit to the size of the cache.
* Issue:https://gitlab.com/BuildStream/buildstream/issues/135
* Developers want to access home directories easily. Sandbox too good.
* Not mounted by default so the user doesn't put their home
directory at risk without their knowledge.
* Per-project project.conf should be able to change the default
* Issue on gitlab needed for accessing the home directory, as part
of the developer experience improvement
* Issue: https://gitlab.com/BuildStream/buildstream/issues/465
* Overlap handling is not intuitive.
* Only a small number of modules overlap (GLib).
* Would be nice for downstream projects to be able to override
which version of a component is used.
* Doc bug for showing how to do it.
* Cross-junction workspaces may solve this.
* Seems a half-solution is pending to be documented at https://
gitlab.com/BuildStream/buildstream/issues/321
* GNOME r-t would like a reply to the proposal at https://
gitlab.gnome.org/GNOME/gnome-build-meta/issues/3#note_115724
* Freedesktop-sdk taking latest versions of dependencies, avoids the
need to provide most security updates. Now the runtime is being maintained we
need a story for providing security updates, since not all upstreams maintain
security updates for old versions.
* This is one of the reasons why the freedesktop-sdk was started.
We need to maintain them somehow.
* ABI stability is a big concern; there is already plans in the
freedesktop-sdk project to tackle this; Javier suggested we have CI on the
release branches to see that the ABI is maintained.
* See https://gitlab.com/freedesktop-sdk/freedesktop-sdk/
issues/108
* Incremental rebuilds (forgot to bring this up during the meeting,
but it's important):
* Issue: https://gitlab.com/BuildStream/buildstream/issues/433

### VCS reduction

* Richard makes a description of the state of the art.
* One motivation is storing failed builds in the cache, and to re-use
previous build trees.
* The motivation for storing build trees in the cache is to support
incremental builds
* Patches for caching the build tree and caching failed builds are
currently blocked on this.
* Issue is not able to re-use VCS history.
* Experiments how to store minimal git history for git describe: not
pretty
* Another option could be to have a SourceTransform plugin to store the
`git describe` output somewhere
* Upstream bug report: for git history reduction: https://gitlab.com/
BuildStream/buildstream/issues/376
* Bug report for VCS history removal as a default - https://gitlab.com/
BuildStream/buildstream/issues/455
* Second issue there was raised in the hope that it would be an easy fix to
un-block the caching build trees and failed build features, but this is not
seen as correct. In fact it was unclear to some people why the second issue
was raised in the first place. To be clear: git history reduction is still
considered the blocker to progressing with the caching features mentioned
above.
* Concerns about reproducibility with caching
* Will this change break 1.0?
* Not ok to have full history in an artifact, ever.
* Conclusion here: we think we can move forward with git history reduction
by attempting it with a shell script - problem is no-one currently able to
work on the issue at the moment. To be discussed outside of the BoF.

### Developer Experience with bst-external

* What is the plan for bst-external.git?
* The current plan is not having official releases ever.
* freedesktop-sdk needs to pick specific sha versions of the packages.
* Who is going to maintain it?
* Jonathan Maw is the official maintainer. James Ennis helps.
* ACTION: James have a conversation with Jonathan regarding
release governance
* Emmet suggests that we raise an issue against the repo and
do this as a plugin by plugin basis if we think that the plugin is stable
* bst-external usage
* Javier to open an issue in the repo and discuss specifics: https://
gitlab.com/BuildStream/bst-external/issues/11

### BuildStream 1.2 release blockers

* Feature Freeze July 30th
* About a month to fix features. Other features go on master
* Required Features
* CAS
* Reduce maintenance on switching from OSTree to CAS
* No duplicate caches in stable and development
* There are discussions necessary between Tristan and Juerg
* Includes
* To provide presentable flatpack developer experience
* Local cache cleanup
* System integration commands
* Split up integration commands into build and runtime
* Staging speed-up for workspaced builds and `bst shell`
* Out of scope or not strictly needed for 1.2
* SourceTransform was suggested as a blocker. It's been through review
cycle, should be close to landing, not considered a blocker.
* License Verification - handeable as a separate tool. We'll label
these as 'Urgent'.

We went off topic a little and discussed maintenance expectations :

GNOME is the active user base whose workflow we will break when we break
things, so we therefore want to get these features into 1.2 so they use
stable, rather than GNOME using unstable master branch, which we do not want.
we need to clarify when we open the 'second door' - ie after the release, when
we get flooded with bugs from GNOME. We need to support 1.2 - GNOME will use
stable, but we should encourage them to help out and use un-stable too.
Message here is: users use stable, if you find a big bug we can fix stable, if
you want to help out then use unstable.

We are talking here about expectation of stability and expectations of core
developers support for stable / unstable - we need a policy / process here for
maintenance and responding to bugs for stable.
Javier: we (GNOME r-t) can setup official stable CI and in parallel another CI
pipeline to test latest bst and provide feedback

### Startup Performance: https://gitlab.com/BuildStream/buildstream/issues/466

* Takes a couple of seconds to start a build
* Need better initial feedback so it doesn't look like it hung.
* Also want to keep the overhead compared to using make down
* Need incremental builds: https://gitlab.com/BuildStream/buildstream/
issues/433
* There was a project between assigned tasks to instrument to find out
where the performance issues were. We got some data that wasn't looked at or
addressed.
* Ideas:
* As much as it's nicer to be able to know what you're going to do
before starting, we need to interrogate caches to see what we already have.
* Looking at the caches is fast if it was done recently because
Linux has its own hot page caches.
* YAML load speed is a known slow point.
* We can't just use libcyaml since we need one that can round-
trip.
* Could keep a pre-parsed cache (like the git index) and update if
inode or mtimes change
* Doesn't need to be a cache of the YAML document, could be
the cache key in some circumstances.
* How are we communicating the user what is happening? Performance vs
communication with users.
* Can we mitigate the effect with first time users through
documentation during the release.

### Buildstream on Windows and OS X

* Current solution: Docker containers
* Proposal: Thin client that uses remote execution for now
* For Windows, run inside WSL; For OS X, distribute with homebrew
* Future: Perhaps use docker/runc/whatever as a sandbox
* We will need GitLab testing infrastructure for this
* Not trivial because we'll need a windows/mac os server
* Alternative: Just make buildstream run natively
* Probably just some groundwork required
* May need to rethink path separators
* Need a distinction between host path separators and sandbox
environment separators
* Note: For performance reasons, we currently concatenate manually
on occasion
* Ensure that nothing else breaks
* Tickets
* https://gitlab.com/BuildStream/buildstream/issues/411
* https://gitlab.com/BuildStream/buildstream/issues/412
* We will need to evaluate both options, native and WSL on Windows.

### BuildStream architecture walk through

* Data model:
* Element contains Sources.
* Elements have dependency trees on more elements.
* In BuildStream Element is a base class of all the plugins.
* Takes filesystem data in and produces other file systems data, and
people run compilers in the meantime.
* Types of Element:
* BuildElement
* We write a YAML file. It runs configure, build, install, strip
commands.
* Stages its dependencies in /, uses commands from
dependencies
* ScriptElement
* Lets you create a layout of where dependencies are set
where.
* You can stage your toolchain in one place, and other things
elsewhere.
* x86 image builder is an example. Gets mkfs, syslinux etc.
then takes a sysroot of contents and makes an image in the output directory.
* FilterElement
* For an element it depends on it can selectively exclude file
contents for its own output.
* ComposeElement
* Runs integration commands on its dependent artifacts.
Intended for making a bootable rootfs.
* ImportElement
* Needs at least one. Takes its sources and puts them in its
output without running any commands.
* Code
* Frontend code
* CLI
* Manages parsing the command-line
* App
* Core
* Stream
* Main API of core.
* Some scheduler details leak into the frontend, currently by
necessity
* Owns the pipeline, loader and scheduler.
* Pipeline
* Provides functions to select elements from a hierarchy
* Except elements
* Main load function
* Loader
* Scheduler
* Contains queues
* Has the `.run()` function for starting an asyncio event
loop.
* Launches a new process for every element.
* Elements run in isolated sub-processes and report their
status back via IPC.
* ArtifactCache
* Artifact insertion
* Artifact retrieval
* Extract caching
* Pushing to remotes
* Pulling from remotes
* Going away in favor of CAS, the abstraction shouldn't be
necessary if CAS supports every platform.
* Sandbox
* Workspaces
* An open workspace is a dump of exactly what would have been used to
stage an element. i.e. the exact commit SHA1.
* When a build is run and staged in a sandbox it is bind-mounted in
rather than staged again.
* Cache keys are calculated differently.
* We use mtime for the cache keys, but we don't share workspace
built artifacts.
* We don't know what the cache key will be at the start of a
build.
* We check the cache key at the beginning to tell if we need to
rebuild. If it's not we unset it and rediscover after building.
* Never pushed.
* In strict mode a cache key is composed of element plus cache keys of
dependencies. In loose mode it's the names of the dependencies rather than
their keys.
* Open issue for the per-element property for whether an element is
strict, it should be a property of the dependency.
* Open issue for the confusing terminology of strong vs strict vs weak
build dependencies.
* There is an open issue about the interaction between weak cache keys and
workspaces.
* Phil Dawson will attempt to draw up the architecture proposal into a
diagram and push upstream - Laurence to send him the email from the ML a while
back where Tristan posted attached his first attempt.

### Other topics

* Sam's bug https://gitlab.com/BuildStream/buildstream/issues/461 has been
discussed after being presented by Valentin.

## Participants

* Agustin Benito Bethencourt - agustin benito codethink co uk
* Alberto Fanjul - albfan gnome org
* Jürg Billeter - juerg billeter codethink co uk
* Laurence Urhegyi - laurence urhegyi codethink co uk
* Valentin David - valentin david codethink co uk
* Thomas Coldrick - thomas coldrick codethink co uk
* Richard Maw - richard maw codethink co uk
* Michael Cantazaro - mcatanzaro gnome org
* Tristan Maat - tristan maat codethink co uk
* Javier - jjardon gnome org
* James Ennis - james ennis codethink co uk
* Phil Dawson - phil dawson codethink co uk

##################

Best Regards

--
Agustín Benito Bethencourt
Principal Consultant
Codethink Ltd
We respect your privacy. See https://www.codethink.co.uk/privacy.html

[Date Prev][Date Next] [Thread Prev][Thread Next] [Thread Index] [Date Index] [Author Index]