Re: Feature proposal: multiple cache support V2

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Mon, 2017-11-06 at 13:58 +0000, Sander Striker wrote:
> On Mon, Nov 6, 2017 at 9:17 AM Tristan Van Berkom <tristan vanberkom codethink co uk> wrote:
[...]
> > So yes, the order being significant is important, however not *that*
> > important afaics, because artifacts which have diverged from mainline
> > are anyway only available in the diverged cache (and whether you
> > download the artifact from one cache or another is pretty
> > insignificant, if the cache key exists, it should be what you expect it
> > to be).
>
> I guess that's true.  I was thinking of the case where the 'stable' cache is more trusted than the 'devel' 
> cache... not so sure that holds unless one is doing signing and the other isn't.

In any case, it's better to have the order significant, if only for the
sake of documenting the behavior more clearly.

[...]
> > I'm not saying I know what the answer is here, but it seems that a
> > global --cache option to BuildStream's CLI is at odds with
> > pipelines
> > constructed from multiple projects - and this proposal should be
> > taking
> > that into account.
>
> That's a good point.  The current user configuration for artifact
> servers allows for both per project overrides (or additions as per
> this proposal?), as well as a global option.

Right so, the global setting does not override but is a worst case
fallback, this could probably be safely removed entirely, although I'm
ambivalent about that.

In any case, the project can decide it's own artifact cache, which is
safer; and the user can *only* currently override this with explicit
mention of the project (so a global setting in the user's configuration
will never deride the choice of a project).

> Are you suggesting that we verify this was the user's intent the
> first time a project is being pushed to the globally set one in the
> user configuration?

I dislike the idea of storing more local state in the project's `.bst/`
directory, and asking the user interactively about things, this seems
to me clunky (if thats what you're suggesting here).

I was more suggesting that if this proposal is going to include
additional CLI options for specifying caches, it should do so with
careful consideration of the motivations behind the current
configuration setup, and also consider that we will very soon have
multiple projects running in the context of a single session (I dont
know what the proposal should *be*, just that if we're going to add
this, it needs to be understood and explained).

That said, it probably makes more sense to discard the additional CLI
options from this proposal entirely, and land the meat of the proposal
as a first step - this gives us more time to breath and consider other
convenience CLI options as a separate topic.

Cheers,
    -Tristan