Re: OSTree and OCI images

[Giuseppe Scrivano <gscrivan redhat com>]

Alexander Larsson <alexl redhat com> writes:

> On tor, 2016-11-17 at 22:31 +0100, Giuseppe Scrivano wrote:
> > Hi,
> >
> > Colin Walters <walters verbum org> writes:
> >
> > > (Email is better for architecture discussions, so moving back here)
> > >
> > > On Thu, Nov 17, 2016, at 05:10 AM, Alexander Larsson wrote:
> > > > Really the only flatpak specific thing is the mapping from
> > > > flatpak arch
> > > > strings to golang/oci arch strings.
> > >
> > > Hum, maybe we can just have "hook" functions that allow the caller
> > > to tweak the content.
> > >
> > > > Yeah, I've started extracting this. The first part is ostree
> > > > support
> > > > for the Docker/OCI layer format (i.e. whiteout handling and file
> > > > overriding), which has a PR here:
> > > >
> > > > https://github.com/ostreedev/ostree/pull/578
> > >
> > > So can we hash out on the list here - what are the advantages
> > > and disadvantages of this "flattening" approach?
> > >
> > > The atomic command right now represents each OCI layer
> > > as an ostree ref.  When we want to deploy a container,
> > > we compute the filesystem tree at checkout.
> >
> > I've recently thought it would be nice to flatten the layers into the
> > commit for the image in atomic as well.  The commit for the image at
> > the
> > moment contains only the metadata (the important part is the list of
> > the
> > OCI layers) used to reconstruct the filesystem.
>
> Due to automatic ostree deduplication I think it would not be
> completely horrible to do *both*. I.e. store the individual layers
> separately, for use when downloading *other* images, and store the
> image itself flattened. You could still store the layer metadata in the
> commit if you need it.

yes, I would still like to keep the individual layers, so that we don't
need to fetch them again when there is another image using them.

> In fact, if we wanted we could store the references to the individual
> layers (by commit id) in the currently unused "related commits" part of
> the ostree commit GVariant. Then we could teach "ostree prune" about
> this so that we can (optionally) make sure not to remove layers that
> are referenced from installed images.

We are already doing this in atomic, if more projects will do the same
then it is probably a good idea to move it in OSTree:

https://github.com/projectatomic/atomic/blob/master/Atomic/syscontainers.py#L850

Regards,
Giuseppe