GNOME.org
 

Re: [BuildStream] Buildtrees in 1.4 release

Hi Sander,

Thanks for taking the time for this thread...

On Mon, 2018-12-10 at 13:13 +0100, Sander Striker wrote:

Hi,

On Sun, Dec 9, 2018 at 9:37 AM Tristan Van Berkom <tristan vanberkom codethink co uk> wrote:

[...]

For this reason it was very intentional that we did not advertise
artifact cache key stability in our 1.0 announcement:

    https://mail.gnome.org/archives/buildstream-list/2018-January/msg00006.html


Sure, but consider the unsuspecting user that only looks at semantic
version numbers.  1.x -> 1.x I expect everything to work as before,
maybe some bug fixes and some additional features, but no surprises
when I upgrade.  Which includes expecting bst build to decide on
exactly the same things to build.
As such it is selective - we chose to make an exception to the
stability rules.  And with that taint (weaken) the meaning of a 1.x
series.


While I agree that the stability guarantees are not complete, I don't
think it was sane to offer stability across the board from that time,
we had to start somewhere.

I hope that we can look into making artifact cache keys stable maybe
even during the next cycle if people think that we can achieve that.


and we should land these mostly simple changes
once and for all in 1.4; what is killing us in landing these features
is not the functionality itself but the tiresome bike shedding which
surrounds them, unfortunately.

 
Can you be specific in what features you consider bike shedding
happening?  And if you recognize this I am assuming you don't care
about the color of the shed?


In this instance bike shedding might be an overstatement, a lot of the
functionality which is landing *needs* a lot of consideration about the
color and shape of the bike shed.


The UI/UX is important, it is what the user sees and interacts with. 
I wouldn't call it a bike shed.  If we would, then we can make that
argument about a lot of things, ranging from coding style to testing
strategies to architecture.  It's all a bike shed, right?
Obviously you care too much to think this, so let's chalk this up to
our communication styles?


Yes, while I agree with your definition of a bike shed more, I have
been seeing it used more generally by some people on the team and
slipped and made an overstatement.

[...]

We should not commit to this *only* based on the fact that it might
currently make a new experimental feature suffer, we should base the
severity of having to make this call on local builds as well.

If local builds do not suffer, maybe we can afford to postpone this
commitment, maybe we can solve it in the remote execution case in some
other way.


If we consider testing we should also consider remote execution in
the same vain.  Because if the parallelism for testing kills the
performance of remote execution there is not much point.


I agree with this, but I don't completely agree that caching a *local*
artifact should affect performance of *remote* execution.

The idea that in a remote execution setting, everything that we cache
locally in a local build setting *must* be cached on a remote CAS
server is perplexing.

If remote execution assumes that everything BuildStream caches locally
should be cached remotely over a network, then I predict that local and
remote execution will always be at odds with eachother.

[...]

In any case, I think you mistook my tone as an aggressive one; when I
say that I am sorry that this distinction escaped me, I do mean it with
the utmost sincerity.


I probably did.  As you did probably take mine.  I always try to read
every post in the most positive light and with the best of
intentions.  Unfortunately, I am only human and sometimes fall into
the trap of reading things in a negative tone.


Agreed.

Frankly, at the time that the remote execution design was taking form,
I was very busy trying to nail down 1.2.x bugfix releases and making
sure that what we had released is not hurting users.

It was easy to overlook that caching something in the local cache,
implied an overhead to remote execution. You on the other hand are very
focused on the remote execution setting, so it is natural for you to
consider the distinction of "uploading" and "caching" the build tree as
nil.

As I mentioned in the previous comment, I am still uncomfortable with
this situation, as it implies that changes we make for local execution
might negatively impact remote execution, and vise versa.


[...]

We have time to complete this optionality, and this optionality is less
expensive than an additional blanket optionality (experimental flag
strawman above) or backing it out entirely, I really think pushing
forward is the smallest effort of any of the choices.

 
Repeat three of this statement.


I think that we agree that the best scenario is to figure out how to
make this optional in the right way and commit to it.


+1.
 

To be frank, it is hard for me to glean from this email what you desire
as an outcome.


Let's just step back and figure out what to do with build trees, to
ensure the feature doesn't have a negative impact.
 

My desire is to move forward without backing anything out, having taken
the time to consider what this means for our future plans first, I
don't want to back anything out and I would *like* very much to have
this optional in the next release - only I want to know exactly how we
are going to tackle this for the testing plan first.


Now that we have bought time by letting go of the release dates, I
think we can figure this out.  My main objection was releasing the
feature with the behavior as-is.
 

That said, all of this talk is secondary to the elephant in the room;
which is that very soon the build trees are *going to be mandatory*
dependencies for elements which run `make check` on the build trees the
necessarily depend on:
 
    https://mail.gnome.org/archives/buildstream-list/2018-November/msg00042.html

 
I still have my doubts that this is the right approach for testing; I
may have only mentioned this in a private setting currently, mostly
for opportunistic/time constraint reasons.  Creating dependencies on
build trees from another element I particularly frown upon.  It's a
dependency type we don't even support yet, and I question whether we
should support it.  However this goes into the subject of testing and
veers away from what we are talking about here.


I think this is the very crux of the issue.

It seems to me extremely obvious, that if we have any chance to
parallelize the builds of reverse dependencies with the `make check`
phase, then we *must* have a way to depend on the build tree of an
artifact from the element which runs `make check` on that build tree.


We should take this back to the other thread.  In short if the
element that runs make check is the same element then there is no
other element to depend on.  This is a different approach to the
whole testing, which is why I mentioned that I still have my doubts
on the current approach.
 

This is *exactly* why making a commitment to this optionality is
dangerous and might be better postponed.


And here we disagree.  Making things mandatory for one feature should
not be detrimental for situations where that feature doesn't even get
used.


I *think* this is a misunderstanding (but am not sure).

If for example, build tree caching is optional except in the case of
caching a failed build, or in the case that the build tree is painted
to be explicitly required in some way: that is different from a blanket
option to disabling caching altogether.

My fear is that by committing to the latter, we turn the former into an
API break.

That said, I agree we should bring this back up in the testing thread.

[...]

Unfortunately, saying later that "turning off build trees" only means
"turning off the build trees which are not required" is very unwieldy
in practice; making the distinction of "a required build tree" is
almost impossible, it is like the having the past be informed by the
future.

 
I really have no idea what you are trying to say here.  In the
hypothetical case that I want to create an element with a dependency
on another element its build tree, I will need to turn on the
creation of build trees for that element.  No time warps necessary.


You speak of that as if it were something obvious.


It is [intuitive] from my perspective.


I believe that when you declare element A which builds something, you
dont want to be bothered about declaring which parts of what you build
are needed by reverse dependencies.

I think it is more intuitive to keep the declaration of whether the
build tree is needed, in the reverse dependency where the requirement
for the build tree arises; I *think* this makes the project data more
clear and easy to maintain.

But again, let's move this over to the testing thread...


However no thought has ever been put into the idea of giving an element
the ability to force it's build tree to be mandatory explicitly, or
whether that is the best approach to ensure the existence of mandatory
build trees.

Maybe a build tree is only mandatory on the host which is processing
the tests in the reverse dependency test element, and maybe the
elements can be grouped together in such a way that the given build
tree is not necessarily ever uploaded to an artifact server ?

This is exactly the kind of conversation I would like to encourage us
to have, a little bit of thought and design towards what we intend to
achieve.


+1.  Let's bring that to a new thread.


Yes, thanks for this :)


 

[...]

If we can answer the question of how to reconcile this plan of build
tree optionality, with the plan of explicitly depending on build trees
in test elements, then I'm sure we can solve this in time.

 
I consider this a blocker for any testing proposal.  Build trees
should never be mandatory, a design assuming this is just
fundamentally flawed.


And I flatly disagree with this.


Sounds like we need to have the discussion.
 

I don't think this needs to be a point of contention though, the
question is only how to have *only* the build trees which matter to the
reverse dependencies which need them, and to avoid overloading the
whole process just because one element temporarily needs the build tree
provided by another.


Let's have the conversation about whether we need the build tree of
another element at all?


I'll try to raise this in the testing thread, but I really believe that
we do, and I even think that no matter how you chop it, it will amount
to the same overhead in the remote execution setting (unless you
serialize testing with builds, and block reverse dependencies from
building while a build is running `make check`, which is sort of the
whole point to avoid).

Cheers,
    -Tristan
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]