[BuildStream] Standardized tests - 2

[Benjamin Schubert <contact benschubert me>]

Hey everyone

TLDR: I propose to add a new optional 'entrypoint' to external plugins to register the sources for the generic source tests.

In [0] I introduced a plan for standardized source tests accross repositories. I have now a slightly different but implemented version I would like to propose.

The relevant MRs are:

- On Buildstream [1]

- On Bst-plugins-experimental [2]

The plan I propose is:

1) Each plugin that want to be tested should define a `buildstream.tests.source_plugins` entrypoint, that contains a `register_sources` method, that does the registration of the sources to run the tests.

Q: Should we keep 'buildstream.tests.source_plugins' or just have a 'buildstream.tests', which _can_ contain a 'register_sources' method? This would allow us to add more registration methods in the future with less effort.

2) When running `pytest --plugins`, we look for all such entrypoints and call it before running the tests.

This has the advantage of not breaking if a specific plugin doesn't implement that, and allows us to extend requirements in the future for other kind of standardized tests.

I also propose to run a fixed version as part of BuildStream CI, plus one job that runs against master but as an indication only.

What do you think? Did I overlook something? Otherwise, please review the MRs and let's get this merged :)

Ben

[0]: https://mail.gnome.org/archives/buildstream-list/2019-August/msg00017.html

[1]: https://gitlab.com/BuildStream/buildstream/merge_requests/1757

[2]: https://gitlab.com/BuildStream/bst-plugins-experimental/merge_requests/60