Re: [RFC] use reflinks to dedup files with the same content

[Giuseppe Scrivano <gscrivano gnu org>]

Colin Walters <walters verbum org> writes:

> > The use case I have in mind is that we will be able to dedup files
> > coming from a container image that are already present in the ostree
> > repository but with a different SELinux label.
>
> Right though...how often does that happen?  If we apply the same
> SELinux labeling policy on import of containers as we use for the
> host, we'll get this for free right?  BTW this is also
> https://github.com/flatpak/flatpak/issues/927

this is not an issue with system containers anymore, as we are going to
use the same SELinux label that would be used on the host.

What I'd like to achieve is to dedup files also for app containers:

https://github.com/containers/storage/pull/137

In this case I think it is preferable to use a different label for these
images but still share the files payload with the host when possible.

This is a first step to achieve what I'd like to do with
containers/storage, it will also be necessary to change how the lookup
in the parent repositories is done and change containers/storage to not
create a bind mount on its storage directory.

> As far as storing the content checksum we compute locally....IMO a
> feature of libostree today is that the storage is so...simple.  It's easy to
> understand and maintain.  As soon as we involve indexed packfiles or
> sqlite or whatever things get less simple.   I think we'd need to do some
> research in this area.

In the PoC PR I've opened, I am using a symlink to store this
information.  I see this feature only as an internal optimization that
should not affect how images are pulled or stored on a remote.

Thanks,
Giuseppe