Re: Trees in volumes

[Colin Walters <walters verbum org>]

Hi, welcome!

On Tue, May 19, 2015, at 07:13 AM, Fabian Deutsch wrote:

> I've been following ostree a bit, the idea, it is actually quite related to what
> we do with [oVirt Node](http://www.ovirt.org/Node), but Node is (basically)
> shipping the trees and their updates in images.
> Because the nature is similar (server side created fstrees) we share some problems,
> like the always-around uid/gid drifts.

Yeah.  So does Docker; see
https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2015-January/msg00008.html

> So, what I wondered - I've brought this up in some smaller cirlces before - is
> if ostree could have another backend, where the trees are not stored in the filesystem,
> but rather in LVM logical volumes with thin snapshots on top providing the writable
> overlay.

This is like https://github.com/fabiand/imgbased right?

> The benefit would be, that the ostrees would not be trees in a fstree, but rather
> that the trees become filesystems in their own.
> This makes a lot of stuff easy, on the host side, because you can mess with the
> filesystem as much as you want.

Right.   I have some thoughts on this from the perspective of BTRFS here:
https://wiki.gnome.org/Projects/OSTree/RelatedProjects

So indeed, a number of things look easier if you rely on a copy-on-write
filesystem underneath.  Docker is definitely the best example of this.  It means
that you can just use yum (or whatever) as is.  And there's a lot of powerful
advantages to that - all the package managers today are well tested and work.

But the thing is, that model doesn't solve some fundamental issues. For example,
I am really quite opposed to the "delete all files underneath running programs"
that all major package managers do.  It can break any nontrivial app that opens data
files while the program is running.

That said I'm sure a lot of people would find OSTree more immediately usable if
they had the "multi-boot" aspect while still keeping the traditional package manager
tradeoffs of "live but racy updates".

Now, one could instead create an offline snapshot, and run apt-get/yum whatever
in that.

I think though the farther one goes down that route, the more you realize what you
really want is *one* program that knows and understands both packages and
the image state.

At least I believe so, and that's where we're going with the
https://github.com/projectatomic/rpm-ostree
project.  

For example, it gives you the package-level diff between the two trees
after you upgrade.

And there's a lot of advanced optimizations we can make and are
just starting to get to, like:
https://github.com/projectatomic/rpm-ostree/pull/146

And finally, what most people want is "package layering" - which is what
you'd have with CoW images and running apt-get/yum "live" on them.  We've
had prototype work on this for some time in rpm-ostree:
https://github.com/projectatomic/rpm-ostree/pull/107

It's been cooking very slowly on the back burner, and you can see there
are some problems to deal with without being able to rely on a CoW
filesystem.  (For example, rpm really wants to live-mutate the rpmdb, so
we make a copy, have it write to that, then rename() into place when done)
We also have to deal with arbitrary %post and make sure they cleanly
"break" hardlinks rather than mutate them.

So I guess what I'd say to this is the CoW idea gets you *very* far quickly -
but then it hits a technological peak where further progress is hard.  I feel
like rpm-ostree (and similar approaches) are going to ultimately be best,
because you really want *one* program that understands the total system
state.

And then people can use whatever they want for block storage underneath
and not have to worry about affecting the package/image system.