Re: [BuildStream] Protect against plugin modifications of artifacts

[Gökçen Nurlu <gokcennurlu gmail com>]

Hi all,

Really fruitful conversations, thanks! I'm also excited to send my
first mail to Apache ML :)

Benjamin Schubert <contact benschubert me>, 26 Haz 2020 Cum, 09:52
tarihinde şunu yazdı:
> What would it affect?
> =====================
>
> A quick look around the plugins shows that the following plugins would be
> affected by such a change:
>
> bst_plugins_experimental/elements/bazelize.py
> bst_plugins_experimental/elements/collect_integration.py
> bst_plugins_experimental/elements/collect_manifest.py
> bst_plugins_experimental/elements/dpkg_build.py
> bst_plugins_experimental/elements/dpkg_deploy.py
> bst_plugins_experimental/elements/flatpak_image.py
> bst_plugins_experimental/elements/oci.py
> bst_plugins_experimental/elements/tar_element.py
> bst_plugins_containers/elements/docker_container.py
>
> So this seems to be more than "a few" plugins and most of those seem to have
> a thing in common: their dependencies are actually part of their output,
> in a transformed way.

+1, I share the same observations

> All the plugins above have a common point: they treat their dependencies as
> their input, as other elements would treat their sources. As a brief summary,
> here is the list of operations they are doing:
>
> - Archiving / checksuming files and writing that in the sandbox
> - Writing a manifest of some kind

Thanks for the investigation, +1. This part is actually my motivation
to chime in. Correct me if I'm wrong, all the plugins above except
dpkg_* ones at least, they are specifically designed to use their
"artifact" outside of the buildstream universe. dpkg_* ones might be,
too. I was just too lazy to read the code in detail..

Here is my 2cs: what about enabling this Directory API
(write/read/etc) only for the kinds of Elements that mark themselves
as "leaf-level" and we forbid them to be used in a different bst file?
Sorry if this has been discussed and rejected before.

At least for the current plugins, they could just add a
`LEAF_LEVEL=True` as a class attr, similar to other flags we have.
For other elements that are actually in the middle of the build plan
and don't have this flag, we can stop exposing
`sandbox.get_virtual_directory()` at all, somehow. I find this easier
to grasp and explain to plugin devs, there'd be less places to modify
in bst codebase and minimal changes for (element) plugin codes.
Citation is needed for the last two though, it's been some time since
I checked the codebase :)

Best,

Gokcen