Re: [BuildStream] [Summary] Plugin fragmentation / Treating Plugins as Sources

[Thomas Coldrick <othko97 gmail com>]

Hi,

> Summary of summary
> ~~~~~~~~~~~~~~~~~~
> So I'd like to hear what people think.
>
> I personally think the venv approach is overkill for our needs, but
> have remained open to it because it also presents advantages, as long
> as it is actually feasible (which I did not believe at all at the
> beginning of this thread, and was optimistic for a time, but now am
> doubtful again after Angelos's assessments).

I'd just like to add my own personal opinion as someone who has written
BuildStream plugins and would like to write more.

> From a pure "What would be the ideal workflow for users and
developers?" standpoint I think that the venv option is preferable. If
for some reason an author did need an external python library it would
be ideal to not have to worry about whether the user has this library
installed at runtime - if it could be handled in BuildStream that would
be great. As a user it would definitely be beneficial to not need to
handle host dependencies in more than one place - both pip and distro
repos. However this is kind of part and parcel of using python tooling.

That said, it looks like this would add a great deal of complexity to
BuildStream internals, and perhaps be built on barely-maintainable
hacks, if it's possible at all.

I think that unless the venv proposal can be done cleanly, then using a
remote-type source is preferable. There are, however, some caveats to
this.

Firstly, I'm not sure if *just* a git type source is suitable.
Expecting authors of plugins for a python tool to use `setup.py` and
pip is reasonable, but forcing a version control system seems more
unreasonable. I'm far from against including a git plugin origin, but I
don't think it should be the only one. To me using something like the
`remote` source as a base seems more reasonable, as long as a hash is
necessary for the loading of such a plugin.

Secondly, if there is no way to safely use external python modules, it
would be nice for BuildStream to provide an API endpoint for checking
host installed python modules, as there is for host-tools. If such a
thing already exists, I must have missed it in the docs.

As a final thought, either way this is implemented it would be nice for
a document on best practices for authoring plugins was added somewhere
in the docs. This would also hopefully mitigate problems like those
experienced with bst-external.

Cheers,

Tom