Re: Feature proposal: artifact signing

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

TL;DR: No. Just No.

On Wed, 2017-10-11 at 18:41 +0100, Sam Thursfield wrote:
> Hello
>
> This is a proposal to add a way of signing BuildStream artifacts with
> one or more private keys, and verifying signatures using the public
> parts.
>
> We propose using GnuPG, via the GPGME library, to calculate and
> verify signatures for artifacts, based on a checksum of the artifact's
> contents.

Yes, this is a feature I very much want integrated into core
BuildStream functionality.

> Storing the signatures should be done in a backend specific way, but
> with a single signing and verification codepath shared by all backends.
> Signing an artifact should not change its identity -- a signature should
> function as 'out of band' data for the artifact.
>
> We propose to keep the process entirely manual from a user point of
> view. This would mean adding new `bst sign` and `bst verify` commands.
> Automatic signing and verification can be implemented later -- first we
> should get the foundations right.

I've read the entire mail, but I'm going to stop right here because
this email is just going to read with a lot of "No".

I dont believe this proposal has been thoroughly thought through, and I
feel that what you are generally suggesting is not a first step towards
something better, but rather leaves us locked in with technical debts
which we can never repay without severely breaking APIs.


I will try to enumerate the numerous problems I have with this:

  o I will not add functionality to BuildStream which needs special
   
care to be of any benefit to anyone at all.

    If a project is configured with a signing policy, it should at
    least require explicit activity on the user's behalf to opt out
    of signing and verification.

    You suggest an initial step which requires fully manual interaction
    in order to leverage the feature, with perhaps some intention to
    improve on this in the future.

    I on the other hand must ignore any promises that "we will some day
    make this better" - the correct assumption is that if something
    lands in master, it will probably never, ever be improved on, so it
    better add value to the default scenarios right away.

  o Out of band transmission of signatures

    When you say:

       "Signing an artifact should not change its identity -- a
        signature should function as 'out of band' data for the
        artifact"

    I think you are proposing to not store the signatures inside the
    artifact storage unit, which I think is impractical for no reason.

    An artifact is a directory with files/ meta/ and logs/
    subdirectoris, we have inherent support for transporting these
    artifacts in BuildStream - and it is not a security concern to
    distribute the signatures of things through the same communication
    channel as the signed data.

    This means that when you look at any big picture of any workflow
    involving signing and verification; you have complicated the whole
    thing just because you dont want to include the signature inside
    the same storage unit as the data it was signing.

  o You make the assumption that there is such a thing as differing
    artifact formats; this insinuates that you would want to introduce
    such a difference.

    I'll reiterate the above, what an artifact is, is a directory with
    files/ meta/ and logs/ subdirectories, and possibly a new directory
    containing signatures of the afore mentioned three - the backends
    we have for storing and transporting them should not effect the
    integrity of the artifact itself.

  o You suggest to leverage functionality of artifact storage backends
    to actually perform the signing and verification.

    I just will not accept the tight coupling of artifacts and how they
    are stored which you suggest.

    If you have received an artifact, and it contains signatures
    of the meta/ files/ and possibly the logs/... then, you will be
    able to verify it with buildstream, regardless of which storage 
    and transmission backend was used before the actual artifact
    ends up in your possession.

    The algorithm for verification will be self contained enough
    that a simple reading of some code should explain what is going
    on.

    How filenames, content, permissions are sorted and serialized into
    a stable reproducible stream and processed by GPG will be clearly
    legible in a single location so that what is happening is very
    clear for anyone who wishes to audit or even reproduce the
    algorithm for the purpose of later verifying artifacts without
    any BuildStream installation present; having a standalone
    python script to verify an artifact would even be desirable.


You are suggesting an approach which first has a lot of manually
futzing around to use, and can possibly have seamless integration later
on, which means you get all the flexibility you want while adding no
benefit to the default use cases.

I will rather consider the opposite approach:

  o Consider a single use case *very well*, lets consider only
    the first step in a potential chain of custody - the automated
    signing of builds by trusted builders and the validation
    of artifacts produced by builders (human or otherwise).

  o Consider all the edge cases of the basics.

    Figure out if you want to error out when downloading an artifact
    which could not be verified; or if you want to emit a warning in
    the context of a build and simply ignore that artifact.

    Consider what happens when pushing an artifact to a share
    which already has the same artifact, which you as a builder
    have contention over it's validity.

  o Integrate that most basic of use cases seamlessly within
    BuildStream, so that for a project which has a signing and
    verification policy, it is enforced *by default*.


This will necessarily not give you the ultimate flexibility that you
might want from day one, judging from your proposal - but if executed
well; it will give us a solid foundation for handling additional use
cases in later iterations (i.e. from here; we can easily expand on the
chain of custody and add more flexibility for sharing of signatures).

Finally, reading over your proposal in general (except for the part
where you want to dig into the intentionally abstract and hidden
details of how an artifact is stored) - You dont need BuildStream to
implement what you want.

All you need from buildstream really is some additional arguments to
the `bst checkout` command, allowing you to checkout a single artifact
addressed by it's cache key, and checkout the entire artifact instead
of just it's files/ subdirectory.

> From there on out, you can do all of the flexible out of band signing
and verification and throwing around of signatures that you want -
there is not really much point to involving BuildStream in this
activity unless it's well integrated.

Cheers,
    -Tristan