On Thu, 2018-06-07 at 14:36 +0100, Sander Striker wrote:
[...]
Right so, we had a bit of a chat on IRC and I just wanted to touch base
on list about this...
Thanks for bringing it back.
Basically:
o The SourceCache by itself will not allow us to really check the box
that "Source Mirroring is supported" (while it does satisfy a
critical subset of those requirements).
Agreed.
o The client side of source mirroring is still interesting, as it
assumes external mirroring solutions exist and just allows
BuildStream to use them.
o I'd rather not start making changes for SourceCache to be a robust
mirror for the subset of features it can provide, until after we
have client side source mirroring solved.
I think they are orthogonal. That said, I would agree that we want SourceCache to remain very simple at first. I guess it depends on what "robust" and "subset" turn out to be defined as.
This means, the SourceCache can remain a cache, and we probably
wont want to change it once we have real mirroring already covered.
Agree.
Interestingly, SourceCache can *still* act as a shared cache for
sources, and possibly optimize the builds in this way.
Indeed.
So, let's not give SourceCache this mirroring burden just yet, lets
finish at least the client side of mirroring first and then re-assess
if we still want that, and why.
*nod*
On a related topic; the last part of this is `bst mirror`, where does
this fit in ?
We need client side mirroring to work with some suitable third party
F/OSS mirroring software project first, and have some user facing
instructions of how BuildStream users can use it... if we cannot get
there, that is when we really *need* `bst mirror`.
If the client side only solution for source mirroring works well enough
without too much fuss then we don't really need the `bst mirror`
service.
The problem however is that it's a pain point for the GNOME build
experience right now, we need it solved in one way or another.
There are some very specific things in the GNOME build setup apparently :) Can we aim to tackle a concrete example?
I mostly want to keep the door open for `bst mirror` to materialize if
it does, because it has the potential of being more convenient to
BuildStream users, and a lot of the code needed to get it done is
already in BuildStream.
On the convenience I'm personally not sold. Essentially this is a setup step [of actual mirror(s)] and a configuration in project.conf for the owner of the project. For the majority of the users it will be completely transparent. That said, keeping the door open seems fine - just as long as we're honest to ourselves about the complexity/maintenance that comes with it.
Cheers,
-Tristan
Cheers,
Sander
