Re: [BuildStream] Merging 'git_tag' and 'git'

[William Salmon <will salmon codethink co uk>]

On 23/01/2020 12:14, Benjamin Schubert wrote:
> Hey Will,
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, 16 January 2020 20:13, William Salmon via buildstream-list <buildstream-list gnome org> wrote:
>
> > Ben
> >
> > I take issue with some of the points bellow but that's not really the
> > trust of what I was trying to say, so I will focus on that.
>
> I am sorry if you feel like that, I did not mean to come out aggressive here.


I don't think you were aggressive i just thought you missed the point of 
what I was saying and focused on technical nitpicks (some of which I 
disagree with) rather than addressing jist of what i was trying to say.


> > I was trying to say, dose the odd additive feature, really pose such a
> > big threat to UI consistency?
>
> For me yes. Let me elaborate why:
>
> 1) When reading an element, it is important to be able to understand
> quickly what is happening in it.
>
> One quick way of understanding that is the element name. Currently,
> if I see 'git' element, I know _exactly_ how it will track, what will
> happen and what I can expect. I do not need to go and read the whole
> configuration of the plugin.
>
> With such changes as proposed here, we would end up having to read more
> for some plugins in order to understand exactly what is happening,
> which puts more mental burden on the person reading it.
>
> Granted, if you only work with a few elements, that is not too much.
> However, depending on the project, you might be exposed to way more elements.
> This also makes reviewing changes much harder, since now you need the whole
> context of the plugin to understand what changed and whether the change should
> be allowed.


I broadly agree with the jist of this.

But I think it means that a project may well decide to use plugins in a 
particular way.

Enforcing the above dose not need to be done by the exclusive use of a 
particular plugin. Isn't it just a matter of policy, and some kind of 
policy enforcement, freedesktop have been working on how to 
enforce/monitor policy, and trying to do it in the open [1] but others 
have also spend effort on how to enforce policy and I believe it to be 
very achievable.


> 2) Documentation. Whenever we add features that are 'optional' on an
> element, we add a new branching point in the documentation. This might be
> added in multiple places, like "So if the branch is that, we do this, unless you
> specify this, in case we do that, unless...". That makes documentation hard to
> read and understand, and adds more burden on choosing the plugin you want.


I agree and believe there is a balance to be made and a skill to writing 
good docs. And there for we should make it as easy as possible to write 
good docks.

But I'm not sure having lots of docs from lots of plugins make it easy 
to maintain or understand them. Wont the plugins docs repeat the same 
text over and over?


> I strongly believe that having more, shorter plugins, with less optional stuff,
> make things easier to understand and decide what options/plugins to pick.
>
> We are advocating for people to write smaller BuildStream elements that
> does one thing, and does it well. I believe we should do the same for plugins.
[...]


I think the rest of your reply broadly follows the quoted text.

The jist as i understand it is that plugins should be short and sweet 
and that its ok to have loads of plugins that all do almost the same thing.

But in my experience of buildstream projects i have found that having my 
plugins, build or source, reflect there underlying tools as closely as 
possible makes it as easy as possible to both build and maintain my 
buildstream files.

And more importantly it makes it easy for a expert in a upstream project 
to help or create there own bst element. While understand the docs of 
one complex plugin will be harder than a very simple plugin I would have 
thought that trying to find the right plugin from many would be harder. 
Especially when the user already under stands the complexity as it comes 
from the underlying tool that they are already familiar with.

The complexity of lots of options, comes from the tool, git has a 
especially complex UI and so it follow that its plugin will be more 
complex than many others. If we have a plugin for every combination of 
options the number of bst plugins will start to scale very badly. This 
multitude of ways to use git also means that a single plugin will be 
more complex and harder to maintain (than for a simple tool) but i would 
argue less so than having many plugins all sharing very similar but 
slightly different code, this mitigated by having a common core but how 
much will really be common?


> I would also have one other question that I am still not clear with:
>
> If both plugins were living in the same repository and sharing the
> bulk of the logic, would you see any other incentives in merging them?


I don't have a issue with there being a few plugins for the same tool if 
its a lot easier to maintain some features separately.

But you seem to be advocating for having many many many plugins for one 
tool.

I don't mind us having one git plugin that dose the bare minimum and 
another that dose exactly the same by default except that it has lots of 
options that can be used in a number of combinations. I just fail to see 
the point of having to maintain both.

It has been brought to my attention that source transforms may be a 
interesting compromise. I don't think they fix all of my concerns and i 
don't think they will work well for some use cases but i do not under 
stand source transforms well enough to be sure.

---

In summary I think these are all trade offs. The Complexity often comes 
from the underlying tool and how we best pass the complexity through bst 
in the least intrusive way possible. I am not advocating from a single 
approach and i think in some case a new plugin rather than a option may 
be called for especially if they is some kind of braking change or non 
optional feature.

But I do think that advocating lots of plugins over options in most 
cases will increase not decrease the burdens of understanding and 
maintainability.

---

[1] 
https://mail.gnome.org/archives/buildstream-list/2020-January/msg00011.html