Re: OSTree and OCI images

[Colin Walters <walters verbum org>]

On Wed, Nov 30, 2016, at 12:29 PM, Alexander Larsson wrote:
> Even if the tarfile had an ostree repo in it i think we would want to
> re-checksum on import. I mean, yes, the thing you import is perhaps
> signed so that you trust it at some level, but if for whatever reason
> it lies about the checksum of an object it could affect another ref in
> the same repo which could come from a different remote (with a
> different gpg key). This seems very dangerous to me.

Right.  Except right now we don't do that for static deltas if:

                                           /* Trust checksums if summary was gpg signed */
                                           pull_data->gpg_verify_summary && pull_data->summary_data_sig,

Which I think is OK for the *host* use case (since content has to be
trusted) but not for the container one.

Ultimately I guess then, the value of having things be "ostree-structured"
on the wire is perhaps smaller.  Particularly given that modern Intel
processors at least have hardware instructions:
https://lkml.org/lkml/2015/9/10/554
(And honestly I bet how heavy ostree is on malloc and the glib atomics
 overhead dwarfs the sha computations in general, but I haven't done
 any analysis)

So maybe we should just drop that conditional and always verify
content even in deltas.

> I don't think you can ever import an image that was designed to be a
> docker image into flatpak. However, you may have built an OCI image
> that targets flatpak without ever being in an ostree repo. For
> instance, your organization may have a build system that combines
> artifacts like pre-built rpms into an OCI image (and adds a metadata
> file). Then we import this into ostree for the first time locally on
> the clients machine during "flatpak install".

Not to get too far into the weeds, but in this model, we're not using
the flatpak "dynamic" runtime/app split?  The app is "fixed linked"
like OCI/Docker?

> Such an image may even be layered, for instance if it was created using
> a Dockerfile.

Right, and if it is - then it seems like this use case ("non-flatpak-oci-to-flatpak")
is going to require the "split layer" model to avoid redownloading layers
for updates, etc.

So I think my vote is to land patches in OSTree ASAP to do the "export
to OCI" case.  We can standardize that mapping of commits -> OCI
(e.g. the json members are "ostreedev.ostree" or whatever).

It's also be obvious to land the import of "ostree-structured" OCI.

What's less obvious to me is whether it's worth adding the
"flattening import" model if every serious use case really wants
"split layer".

(You did make an argument that perhaps we want both, to make
 checkout more efficient...but I feel like unless there are a *lot*
 of layers or a *lot* of whiteouts it's going to be a wash)