Re: Proposal: JSON for artifact metadata instead of YAML

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Wed, 2017-11-01 at 15:53 +0000, Angelos Evripiotis wrote:
> Hi Tristan,

[...]
> > If we introduce JSON in a part of our public API surface (which
> > artifact metadata will inevitably become if we make artifact format
> > stable and expose it to external tooling), that means that projects
> > which consume BuildStream will have to use multiple parsers and
> > understand multiple formats to interact with BuildStream and script
> > around it, forever.
> >
> > This significantly detracts from the simplicity / ease of use of
> > BuildStream as a whole, and as such it is a highly undesirable outcome.
>
> I would agree for many formats, in the case of YAML and JSON I think we're in
> luck. It turns out that JSON is YAML, so all YAML 1.2 readers can also read
> JSON:
>
>     "The primary objective of this revision is to bring YAML into compliance
>     with JSON as an official subset."
>
>     http://yaml.org/spec/1.2/spec.html (from late 2009)

I didnt know this, very interesting.

I still think it's desirable to have things look consistent, especially
if/when the artifact format goes public and people read the files - but
this effectively means we can switch from one format to the other
without ever breaking any compatibility of tooling, so long as this is
always advertised as YAML and not JSON (external tooling which assumes
JSON would break when we "fix" it to be consistent later on), it's a
good thing to know.


[...]
> > Doing things "nicely" is "hard", yes - is it so "hard" that we have to
> > sacrifice "nice", permanently ?
>
> I think we share the same intent to put in the hard work to make sure
> BuildStream is done the "nice" way.

Sorry for the way that comment may have sounded, i didnt mean to
insinuate anything by it.

In general, at a macro level I think it's correct to design along the
path of least resistance; when it comes to the smaller details and the
walls we hit along the way, I tend to prefer bending the world to my
will rather than making compromises to an already designed API surface.

> From a design perspective, the quality of the outcome depends immensely
on how consistent the API is, how simple it is to understand, i.e.
everything that makes it attractive and easy to use is of paramount
importance - if the plan makes sense at all, surely in time there are
ways to optimize the implementation backing it up.

Once we start making the first sacrifices of API consistency on the
alter of optimization and mere implementation details, it becomes a
very slippery slope from what was once a sound design, towards a huge
incomprehensible mess.

Cheers,
    -Tristan