[BuildStream] min-version approach for plugins

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi,

  As per our BuildStream 2.0 game plan[0], I started work on enhancing
format minimum version requirements and ensuring proper messaging when
mismatching BuildStream versions (1 vs 2), i.e. see "min-version" in
the referenced email[0].

I encountered a bit of a conundrum however when looking at plugins and
how they fit into the picture, with regards to BST_FORMAT_VERSION[1]
and how these version guards are checked in the pip plugin source[2].

As Chandan is mostly "in the know" for most things "pip", I pinged him
and we had a discussion on IRC today to try to find a way forward[3].

This is what we came up with:

  * For local plugin sources, or plugins which are loaded from across
    a junction (not yet implemented, also discussed in [0]), it is not
    important to have versioning guards.

    Rationale:

      These plugins are very precisely referenced by the project which
      loads them, as such there is no room for variance or unexpected
      plugins or behaviors.

  * For pip source plugins, we should allow the project.conf to specify
    semantic version specifiers as defined by pep 0440[4], and require
    that plugin packages to be loaded via the pip plugin source follow
    the same pep 440 for its version identifiers.

    Rationale:

      These are much easier for users to express and understand than
      the BST_FORMAT_VERSION numbers we were using previously, it is
      also less work for plugin developers as they do not have to
      remember to bump a version digit on their plugin classes as they
      evolve.

      Users will have errors saying "project required foo >= 2.4, 
      but only foo 2.2 was installed" (or similar), rather than
      messages saying "project required foo plugin format version
      26, please go figure out what package version you need for
      yourself", making life easier all around.

To implement this for pip plugin sources, we can use distlib[5] and do
not need overly complex code to do it.

I think this will be a net improvement over BuildStream 1, where I have
been seeing lot's of breaking/changing plugin API surfaces evolve, and
have yet to see the BST_FORMAT_VERSION of a plugin set to anything
other than 0.

Cheers,
    -Tristan

[0]: https://mail.gnome.org/archives/buildstream-list/2020-April/msg00013.html
[1]: 
https://docs.buildstream.build/master/buildstream.plugin.html#buildstream.plugin.Plugin.BST_FORMAT_VERSION
[2]: https://docs.buildstream.build/master/format_project.html#pip-plugins
[3]: https://irclogs.baserock.org/buildstream/%23buildstream.2020-04-21.log.html#t2020-04-21T10:41:17
[4]: https://www.python.org/dev/peps/pep-0440/
[5]: https://pypi.org/project/distlib/