Re: Stop the train ! Caching build trees is going to be too big

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Fri, 2018-04-27 at 11:25 +0000, Sander Striker wrote:
[...]
> > What I propose that we do, is the following:
> >
> >   * Split artifact keys in two:
> >
> >     * The regular artifact remains "${project}/${element}/${key}"
> >
> >     * The cached build tree is addressable as
> >       "${project}/${element}/${key}/build"
> >
> >     * Alternatively, we split the artifact into metadata, logs,
> >       output and build components, this remains to be discussed
> >       and analyzed.
>
> I would prefer us to take a slightly different approach than storing
> under different keys, and instead store a "BuildStreamArtifact"
> message under the key.  That can then be used to download
> the different elements of the artifact.
> This is a similar approach to the ActionResult stored in the
> BuildFarm ActionCache.

The only real problem I have with the approach you suggest is that it
sounds like it can only be implemented with the CAS artifact cache,
while implementing it the way I suggest would probably be transparent
when we change artifact cache implementations from OSTree -> CAS.

Really this is only a matter of making it easier for separate ongoing
developments to happen without friction or blocking on eachother, and
this also sounds like an implementation detail which we could change
without too much effort later on, at the cost of an artifact version
bump.

Does this make sense ?

Cheers,
    -Tristan