Re: Proposed public data fields for dpkg build and deploy elements

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Fri, 2017-07-07 at 10:42 +0100, Jonathan Maw wrote:
> On 2017-07-07 09:12, Tristan Van Berkom wrote:

[...]
> Hi Tristan,
>
> Currently, it is impossible to generate a "devel.deb" package because 
> there's no
> "control" metadata for the "devel" split domain.

Ahhhhhhhhh, I reread your original mail and see that I missed how
complex this actually was at first sorry.

So, to be honest I'm not satisfied with this for the long term; but I
can see that it would work, given that you have previously extracted
the control metadata from a 'dpkg-build' element which had that
metadata.

However it leaves hanging the cases where we want to build debian
packages without much screwing around with the metadata, one has to
encode a bunch of debian specific stuff in there.

Also it means we dont share as much as we potentially could; for
instance if I want to take the same element and have it deployable both
as 'deb' and 'rpm' packages, I will have to replicate a bunch of stuff
to get it done (and I think that is the more interesting use case
generally speaking: One set of build metadata to use for multiple
deployment tactics).

So ultimately what I think we should be doing:

  o Split up the different fields from the control file which may have
    been parsed at 'dpkg-build' time, into separate 'package',
    'dependencies', 'architecture' and 'description' attributes.

  o Understand what the 'dependencies' actually are at 'dpkg-build'
    time to populate the metadata more specifically

    This is so that we can have a 'dependencies' field which is a list
    of words on the target system, not things like ${shlibs:Depends}

  o Generate the control file at 'dpkg-build' time ahead of building
    the package, based on attributes

  o Default to the assumption that an element will be deployed on
    a debian system where the element's dependencies have also been
    deployed on the same system; i.e. in the absence of manually
    defined dependencies, we should be able to cook this up based
    on the element's non-recursive Scope.RUN dependencies.

However !

This is probably too much for an initial implementation and will
require much more experimentation, in order to automatically deploy
regular build elements without overly specified metadata to debian
packages.

So for this control file, lets keep in mind only that we will in the
future want something much better, but that we are going to have to
remain backward compatible with element formats which specify this
'control' attribute.

I suppose that in a future with more granular metadata and more
automation, the presence of a 'control' attribute in 'dpkg-data' would
have to override any automation that the 'dpkg-deploy' element would
likely do.

> I see what you mean, though. In future, if we want to generate debian 
> packages for
> ordinary elements, splitting them along -devel, -doc and -debug lines 
> would require
> a fair amount of manual effort to re-define the split-rules.
>
> Are we interested in generating packages that aren't prefixed with the 
> element name?
> i.e. python3-defaults-debian produces the idle3 package. Would we still 
> want to generate
> idle3 from python3-defaults-debian, or would we be expect 
> python3-defaults-debian-idle3?

I think that in this case we should at least default to a package
named: "<element-name>-<split-name>.deb".

In your metadata you have:

  public:
     bst:
       dpkg-data:
         foo:
           control: |
             Package: bsdgames
             ...

This could generate <element-name>-foo.deb by default, and then
something could be used to override the package name, if that is
desirable, i.e.:

  public:
     bst:
       dpkg-data:
         foo:
           name: full-package-name-manually-overridden

Also, I'm not sure what "Package: bsdgames" is doing in there, but I
suspect that it might just have some kind of influence on the name of
the package being generated.

If so, can we at least start by splitting this "Package:" portion of
the control metadata out of the rigin "control" attribute and have that
separately from the beginning ?

Cheers,
    -Tristan