Re: Project options and other format enhancements (and dropping "variants")



On 16/09/17 07:22, Tristan Van Berkom wrote:
On Fri, 2017-09-15 at 15:53 +0100, Sam Thursfield wrote:
Hi Tristan,

In general I agree with your premises, and I think the proposal is
workable. I don't have anything better to propose.
     Option Declaration
     ------------------
     A project declares which options are valid for the
     project in the project.conf.

     These options should have some metadata which can be used
     to declare the defaults, assert valid values of the options,
     and also a description string which the CLI can use to communicate
     the meaning of project options to buildstream users (not all users
     building a project wrote the project.conf).

Are you expecting to support only enum values, or freeform strings and
integers too?

I certainly dont want to support only enums, although I was undecided
on whether the data would simply be in string form and have conditional
statements deal with typing; or to have typing encoded into the option
metadata.

I would much prefer typed values. Otherwise you end up in the land of "do I turn off this feature with 'no', 'false' or 'off'" ?

I would also like to avoid or at least strongly discourage having freeform strings. From an analysis point of view, an enum option with 15 possible values represents 15 possible variants of your build. A freeform string option represents effectively infinite variants of your build. (In small projects you may be able to drill down and see how many string values are actually used, but in huge projects that becomes impractical).

One idea I am liking more and more is the (contains list_opt "string")
kind of conditional, where one could check for the presence of a word
in a whitespace separated wordlist, which one could use to whitelist
elements for a feature or to test for a single feature in a list (like
the compiler tuning example I made in my reply to Sander).

You probably already made the connection between this idea and Gentoo's USE flags: https://gentoo.org/support/use-flags/

It's a good system provided the possibilities are limited. So the option definition should have to declare all the possible flags that can be used in the list, rather than it just being a freeform string.

<slight deviation from topic>

As you've been looking into bootstrapping compilers with BuildStream
maybe you can shed some light on what we could do for this, because I
feel my approach doesnt solve it perfectly.

 From what I understand, currently we can only single case symbolic
machine names and make a huge list of full tunings flags depending on
that symbolic name. This is an area I think yocto excels at and I would
like to have a solution that allows enough flexibility for this (of
course without being shell scripts which execute and source eachother).

At the basic level, maybe this could be done by allowing a project to:

   A.) Define symbolic names, maybe they are "macros" or "presets"
   B.) The symbolic preset defines values for options
   C.) Write conditionals based on the options

It's just the first approach that comes to mind, but would allow us to
define feature lists associated to symbolic machine names, and then
write conditional YAML fragments based on the resulting feature sets
instead of having to special case every machine name individually.

Any other ideas ?

Implementing a solution to this should not block our landing of a
project options feature; however, our approach should probably be
informed by if/how we intend to address this kind of complex case.

My goal with architecture names in Baserock has always been to limit the possibilities to a fixed set that we can exhaustively test. So in a way it's a feature rather than a bug that the way to enable extra project-wide compile flags is to modify gnu-toolchain/gcc.bst or project.conf directly.

I'm fine with people setting random GCC tuning flags if they want, but it should be be clear that if they do this then they have diverged from the upstream reference builds and any breakages are for them to fix, not us.

So I don't think we need any complex solution here. However there are certain variations where we probably do need to support both upstream and for those encoding info in the architecture name sucks. The only current example I can think of is ARM hard-float versus soft-float.

Having a "CPU architecture" flag set which can contain specific flags based on the overall symbolic architecture name seems like the nicest approach there. So if your arch name is "armv8l64" you get arch_flags=hard-float and you can override that to arch_flags=soft-float if you wish. If your arch name is "x86_64" then neither option is valid (or perhaps more accurately 'hard-float' is the only valid option).

</slight deviation from topic>



As I've said else where I've migrated from:

   '??':
     condition:
     ...

to:

   (?):
     condition:
     ...

I feel like it will stand out more and I dont like the quotes. That
said I'm open to changing the conditionals to something more
conforming, if we really expect that the result is going to be more
legible.

I don't like the quotes either, (?) is probably better.

Ok so frankly I'm not attached to parenthesis nested lists esthetically
  speaking, I am however quite attracted by the simplicity of it, we
probably can achieve similar simplicity with something else.

I'm not convinced that a:

    condition: |

      %{foo} == "bar"

kind of notation is simple though; it tries to be very human friendly
and programing languagey, and then leaves us a bit blind if we want to
later extend the operators, what would we use for the kind of word in
list 'contains' conditional ? (maybe we do like python sets and use C
bitwise field test operators on them ?).

This operator comparison expression approach is also more rigid and
demanding, it would have to be done perfectly the first time.

We dont have a nice relaxed namespace in which we can deprecate the
'ifeq' symbol for a new 'equals' symbol on the day that we figure out
that comparisons should have been case insensitive, we might instead be
in a corner left with yucky workaround alternatives, advising users
that they should use the '===' operator in new projects, instead of the
existing but botched and unrecommended '==' operator.

I think deprecating "ifeq" for "equals" is pretty horrible too. We should just get it right first time, whichever approach we choose :-)

I agree the simplicity of s-expressions and nested lists is a strength, but I still think it comes at a usability cost. Also, just because it's simple now doesn't mean that it won't have grown to be monster in 2 years time because everyone who uses it gets excited and sends in a patch to add their favourite language construct.

What I'm suggesting is that we avoid creating a new DSL for expressions in the first place, we try and reuse an existing one instead. Ansible's use of Jinja proves that this is possible. I'm not entirely sure how it works though -- if I get time I will try and make a proof of concept expression parser and we can see if it can beat your 218 lines of code :-)

Sam

--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]