Re: [BuildStream] Source cache summary

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Ah, missed the summary...

On Fri, 2019-02-01 at 15:58 +0000, Raoul Hidalgo Charman via BuildStream-list wrote:
> Hi everyone,
[...]
> For source cache, the plan is for the class to be in the context, such
> that the `Source` class calls it before fetching sources directly from
> upstream.

Ok so I missed this part from previous discussions.

I'm not sure we have the right stacking order here of modules, I see
two ways this can make sense:

A.) SourceCache manipulated by the Source

    In this scenario, the code flows much the same way as it currently
    does, but some internal `Source` methods now have knowledge of
    the SourceCache and delegate some calls to SourceCache.

    In this scenario, the `SourceCache` does not know what a `Source`
    is, and does not delegate any functionality to a `Source`.

B.) Source manipulated by SourceCache

    In this scenario, the core Element routines which currently talk to
    Source instances, instead talk to the SourceCache directly, passing
    it along a Source which it can use to both identify the right key
    for the staged payload, and also to fall back on to call methods
    when the sources are not in the cache.

    In this scenario, the `Source` does not know what a `SourceCache`
    is, and does not delegate any functionality or ever call into
    the `SourceCache`.

My understanding from your previous emails was that `SourceCache` would
 implement certain Source manipulation methods and delegate those to
the real Sources when the cache could not provide data (i.e. basically
B), but now you are saying that you intend for the Source to *call* the
SourceCache, this is confusing.

>  This would mirror the way elements call the artifact cache
> before building the artifact.

Ok so I kind of like the symmetry, in which case the SourceCache only
takes a Source in order to easily construct keys, and to do logging,
but never calls methods on the `Source`.

Do I understand that with this summary email, we have dropped the idea
of having the `SourceCache` have all the methods which `Source` exposes
and having it call into Source ?

In any case, please clarify the stacking order of the Source
SourceCache modules and please try to ensure that stacking is vertical
and we don't introduce circular codepaths between modules.

Cheers,
    -Tristan