Re: [BuildStream] Migration of plugins away from BuildStream core

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi Jim,

Thanks for raising this.

I think that we would be putting the cart before the horse if we were
to proceed with moving plugins outside of the core without solving the
issues around:

  * Making it easier to use plugins outside of the core

    * As you pointed out, while using the `pip` technique does seem to
      work for me most of the time; when it doesnt work, it fails
      miserably.

      Incidentally, someone who was present at the meeting about
      externalizing plugins at BeaverCon in Manchester 2018, had such
      an issue installing `bst-external`, and we were unable to quickly
      fix it.

    * I had given this some thought last year, and brought up some
      ideas on how we might be able to treat external plugins as
      git repositories, and have BuildStream core include a special
      `git` based Source specifically for obtaining a project's
      required plugins.

      https://mail.gnome.org/archives/buildstream-list/2018-November/msg00065.html

  * Making it easy to maintain plugins

    * Exposing parts of the test suite which make sense.

      E.g., we have a whole battery of tests which run against every
      "source kind", these are currently scattered across various tests
      but when combined into one invokable suite, constitute a test
      case asserting our expectations of abstract sources and how they
      should behave.

      An external plugin repository should be able to implement a
      'Repo' scaffold (tests/testutils/repo) and add a one liner
      to their own test suite in order to run BuildStreams thorough
      source tests against their sources (asides from additional custom
      tests which cannot be generalized, which need to have separate
      tests regardless of where the plugins are hosted).

    * Expose some API for building plugin documentation.

      We have some custom mechanics for building documentation already,
      if we can wrap this up in some way which plugins can use, then
      we can guarantee consistent documentation style across plugin
      repositories.

      Moreover, we can have the liberty to *change* these mechanics
      in the future in the main BuildStream repository, so any
      stylistic changes should propagate naturally to plugin
      documentation.

      Getting this right at first allows us to dodge a huge mess
      later down the road, or a tiresome game of whack-a-mole where
      we make efforts to hunt down plugin maintainers and patch those
      modules to produce consistent documentation.

  * Solving the CI problem in general

    * With separately moving repositories, we need plugins to test
      against a stable version of BuildStream, and have a way to
      upgrade.

      Similarly to how we always control the set of dependencies in
      a CI environment so as to ensure that tests can only fail because
      of code changes in the MR under test.

    * Similarly, losing the plugin tests from BuildStream core means
      that we miss out on plugins which exercise core BuildStream
      codepaths which still need to be tested as a part of CI.

      So in the same way as plugins need a stable version of
      BuildStream which is upgraded periodically in CI, BuildStream
      also needs to install and test against a limited set of plugins
      known to exercise all of BuildStream's internal codepaths.


On Tue, 2019-02-26 at 18:08 +0000, Jim MacArthur via buildstream-list wrote:
[...]
> I've now read all the above sources, and all the documents linked from 
> your first message, but not anything linked further than that; this has 
> taken me about 30 minutes so far.
>
> Nothing yet in there says why we're planning to move plugins out of 
> core. The only one that hints at it is Agustin's summary of the 2018 
> Gathering, which quotes:
>
> "BuildStream doesn't want to keep all that baggage"
>
> ... and that's it, that's all the justification I can find. Note that 
> this document also says:
>
> "Second reason is for keeping plugins that are not yet mature enough 
> [...] but eventually want to move into the BuildStream core, e.g. 
> X86Image". This directly contradicts today's plan; we are planning to 
> move mature plugins out of core.
>
> Also: "BuildStream cannot maintain every plug-in that people write, we 
> cannot bring them all into core. But we should have a convenient set of 
> plugins distributed with the core"... which I fully agree with. It then 
> explains some of the pain users have already reported from having to use 
> bst-external.

I think these summaries are incomplete and inaccurate.

We did discuss this at length at the 2018 gathering (in Manchester) at
length, so I'll try to summarize what I can from memory.

* We want to avoid a situation where everything and the kitchen sink
  ends up in BuildStream, and encourage more development and
  maintenance of plugins outside of BuildStream.

  If BuildStream itself becomes a home for "all the plugins" then we
  sort of defeat the purpose of having plugins (not completely, but I
  think it would be counter productive to consider BuildStream to be
  a home for all plugins).

* There is already an existing misconception that plugins should
  graduate from external projects into BuildStream.

  This is a social problem, there is an unfounded desire to get your
  plugin accepted into the "golden upstream" set of maintained core
  plugins.

  This lead to discussion about where we should draw the line,
  and what the criteria should be for accepting a new plugin into
  BuildStream core.

* I think that expensive technical solutions to social problems
  should be considered beneath us as a rule, and I personally
  do not care in the slightest about hurt feelings of developers
  who may feel discriminated against just because their plugin is
  not accepted upstream.

  I would be happy with a statement that the existing plugins
  included with the core exist for historical reasons, and to
  encourage people to maintain plugins outside of the BuildStream
  core, and/or use custom plugins if the core set of plugins
  do not satisfy a given use case.

  I don't think the majority of contributors in the room at the
  time agree with my stance on this, though.

* I do agree that in an ideal world, we would have a clearer story
  about what plugins should be included in the core.

  This led to discussion about what that story would be.

  Incidentally, we also discussed a gstreamer-like approach where
  we could have -good/-bad/-ugly repositories, however it was also
  raised that having a "bst-plugins-good" kind of repository would
  result in the same social problem where different parties are
  competing to have their plugin "blessed".

* In the end, we did agree to move the majority of plugins outside
  of the core, excepting only for "special" elements which are either
  very generic or are special due to internal relationships with the
  core.

  E.g. `filter` and `junction` elements need to use private API to
  interact with the core because of their special natures, as such
  we can never move them outside of the core.


While I do think we were all in agreement about moving these plugins
outside of the core, I don't recall any agreement about recklessly
proceeding with this plan in advance of developing the required
infrastructure for this to be a painless experience for both users and
plugin repository maintainers.

I do recall explicitly punting this change until after the "next
release" at the time, *because* of the logistic overhead involved in
making this painless.

My recommendation would be to create a single test plugins repository,
and use this as a model to solve all of the issues outlined at the
beginning of this email. When we are perfectly satisfied that the
maintenance effort for plugin repositories is sufficiently low, and the
user experience is not negatively impacted (if not improved) by moving
plugins around, *that* would be the time to actually start migrating
plugins around.

Cheers,
    -Tristan

> Incidentally I've spent most of today trying to get bst-external 
> installed inside a docker container so I can use the quilt plugin. It's 
> a long way from what Phil Dawson describes as "trivially installable".
>
> I agree with what Mathieu Bridon has said: Without things like git and 
> tar, BuildStream does not do anything useful.
>
> Jim
>
> _______________________________________________
> buildstream-list mailing list
> buildstream-list gnome org
> https://mail.gnome.org/mailman/listinfo/buildstream-list