Re: OSTree and OCI images

[Alexander Larsson <alexl redhat com>]

On Wed, 2016-11-30 at 14:27 -0500, Colin Walters wrote:
> On Wed, Nov 30, 2016, at 12:29 PM, Alexander Larsson wrote:
> >
> So maybe we should just drop that conditional and always verify
> content even in deltas.

I think that makes sense.

> > I don't think you can ever import an image that was designed to be
> > a
> > docker image into flatpak. However, you may have built an OCI image
> > that targets flatpak without ever being in an ostree repo. For
> > instance, your organization may have a build system that combines
> > artifacts like pre-built rpms into an OCI image (and adds a
> > metadata
> > file). Then we import this into ostree for the first time locally
> > on
> > the clients machine during "flatpak install".
>
> Not to get too far into the weeds, but in this model, we're not using
> the flatpak "dynamic" runtime/app split?  The app is "fixed linked"
> like OCI/Docker?

Oh, no, we can't change *everything* just because we ship things
differently. It'll just be exactly the same files in the tarballs as
would be in the ostree commit. So something like:

 metadata/
 files/bin/the-app

So, clearly such images would have to be created using custom code
that knows how the final layout should look. Much like how an image for
a host-ostree tree would need a special layout.

> > Such an image may even be layered, for instance if it was created
> > using
> > a Dockerfile.
>
> Right, and if it is - then it seems like this use case ("non-flatpak-
> oci-to-flatpak")
> is going to require the "split layer" model to avoid redownloading
> layers
> for updates, etc.

It doesn't *require* it, but yeah, if that case is common, then it
would be an important performance optimization.

> So I think my vote is to land patches in OSTree ASAP to do the
> "export
> to OCI" case.  We can standardize that mapping of commits -> OCI
> (e.g. the json members are "ostreedev.ostree" or whatever).
>
> It's also be obvious to land the import of "ostree-structured" OCI.

An export of any random OSTree branch would result in a single-layer 
OCI image with some metadata from the commit id.

Such an image would be imported with the reverse operation.

But, what do you expect to happen if we try to import a multi-layer
OCI image (i.e. one that wasn't created by an export). Should we just
fail because its in the wrong format? Or should we create multiple
commits, one for each branch, and then some special kind of commit that
ties these together? How would a user then know how to check out such a
ref? Does he have to parse the OCI json and do the mapping to the other
branch names for the layers? 

Here is where having a flattened import helps a lot, as it makes it
very obvious what the answers are to these questions.

> What's less obvious to me is whether it's worth adding the
> "flattening import" model if every serious use case really wants
> "split layer".
>
> (You did make an argument that perhaps we want both, to make
>  checkout more efficient...but I feel like unless there are a *lot*
>  of layers or a *lot* of whiteouts it's going to be a wash)

Its not necessarily about being efficient, but about having a way to
represent the composite image as an object in native ostree terms,
without the user having to know about the inner OCI structure. You can
refer to it easily, you can inspect it, check it out, all with regular
ostree operations.