Re: [BuildStream] Protect against plugin modifications of artifacts

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi William,

On Wed, 2020-06-24 at 13:52 +0100, William Salmon wrote:
[...]
> > If this is such an interesting feature to have, I don't see much reason
> > why we could not implement such a feature in BuildStream, even using
> > Python plugins. This could be an Element API which takes a Sandbox, a
> > "%{variable}" name and an absolute path, which could stage a resolved
> > variable as the content of a file in the sandbox safely.
> >
> > It would be interesting to see a proposal for such a feature, probably
> > I would argue that the permissions used to stage such a file be very
> > limited, or unspecified (something matching the hard coded permissions
> > used to stage files from Sources into the sandbox).
>
> I don't strongly disagree with the jist of this. Things like `with 
> conditionally resolved variables` seems like it might hint at the 
> required flexibility.
>
> > That said, a controlled feature like this would be an extremely far cry
> > from allowing python code to simply write whatever they want into the
> > sandbox, and would not allow for the non-deterministic things which are
> > currently being done by existing plugins which exploit this currently
> > existing weakness.
>
> In fact I have said in this thread:
> """I am not saying that this API should not be improved or that there is 
> no room to make it better"""
>
> I would think that we can come up with something that would be enough 
> for many of the EXISTING AND IN-USE plugins that have been created and 
> used in good faith that need to put things in to the sandbox. As well as 
> supporting the plugin's that should exist like a genimage plugin.
>
> I look forward to getting the details sorted so we can be sure they are 
> useful and reproducible.
>
> I don't think its a good idea to remove the old API without adding the 
> new API as this would brake and block many existing projects who are 
> trying to track bst-master and are providing feed back and many bug 
> reports and some MR's.

Right, the thing is, I suspect that plugins which are exploiting the
ability to write directly to the sandbox, which could be implemented
with the above kind of suggested API, are the minority (but I've only
so far looked at collect_manifest and oci, which are both rather
hopeless in this regard and need a complete rethink).

> From an upstream perspective, I think it's important to close the
floodgates ASAP, and consider any (controlled/limited) API for writing
out variables to files separately.

Further, it would have to be *ensured* that these contents cannot be
modified with python, that's why a variable name would be just about
the only thing I could imagine would be workable.

In any case, I think it's important to untangle these two separate
topics, what we have been discussing and what the main focus of this
topic is, is the ability to write to the sandbox, and the implications
of this, which needs eliminating - what plugin authors do about it for
the various plugins is a separate topic, and may involve adding
appropriate helper features to BuildStream.

Cheers,
    -Tristan