Re: Running an ostree deployment in a container

[Arnaud Rebillout <arnaud rebillout collabora com>]

Hey !


On 04/26/2018 04:16 AM, Colin Walters wrote:
> Hi,
>
> On Wed, Apr 25, 2018, at 6:22 AM, Arnaud Rebillout wrote:
> > Dear OSTree maintainers,
> >
> > I'm attempting to build a basic Debian Stretch OS with OSTree. 
> Cool.  Is this for a server use case?  Desktop?  Embedded?  The answer
> to that informs a bit how you do testing.

Desktop, GNOME based.

> > So far so good, I end up with my "deployment" directory, and now I would
> > like to test it in a container or a VM.
> >
> > I noticed that the man page of 'systemd-nspawn' gives an example for
> > running an OSTree deployment [1]. Plus, Philip Withnall added the
> > '--pivot-root' option in Feb. 2017 for that purpose [2]. So I thought
> > I'd give it a try.
> I haven't played with nspawn for this much personally.  In rpm-ostree
> we actually run each rpm script (%post) etc. in a bwrap container
> inside the target root.  
>
> The code is mostly here:
> https://github.com/projectatomic/rpm-ostree/blob/1521dc32c9c741b9050b5348a86f3500eea6e62c/src/libpriv/rpmostree-bwrap.c
>
> That said this is probably more useful for a quick test:
> https://github.com/projectatomic/rpm-ostree/blob/master/scripts/bwrap-script-shell.sh

Thanks a lot for the links. I hear a lot about bwrap but I didn't tried
it yet, maybe now is the time :)

> > And, to be a bit more general, what's the guidelines to test a
> > deployment locally? Looking at containers, systemd-nspawn seems to be
> > the way to go, it's a one-liner (almost). 
> One reason I prefer bwrap over nspawn is that bwrap is nestable (container in container)
> a lot more nicely.  This has come up in our CI, which uses Kubernetes.
> We haven't yet plumbed everything through there to make it work, but
> it's on our near term radar.  (Actually the main blocker right now is our
> SELinux policy denies e.g. mount() inside containers today)

Good to know, thanks for sharing.

> > Looking at virtual machines
> > however, I struggled a lot with QEmu, but didn't come up with a
> > straightforward solution to create a bootable image out of my deployment
> > directory. Not yet. But maybe I just need to learn my qemu-fu better ;)
> Most of our testing for Atomic Host today rebases/upgrades existing cloud
> images.   You could almost certainly take e.g. a FAH cloud image and add 
> a remote to your repo, and `ostree admin switch` to it.   (rpm-ostree
> currently barfs on trees which don't contain an rpmdb, we may fix it someday)
>
> That opens the question though to how one generates a cloud image/ISO/etc.
> For Fedora-derived OSes we use Anaconda which supports ostree now.
> In fact, it's highly likely that you could use Anaconda to install your ostree
> content, even though it's Debian; I tried to support that use case theoretically,
> though see:
> https://github.com/rhinstaller/anaconda/issues/1259#issuecomment-350272927
>
> You can also probably dig a bit and find the different image building tools
> from the various libostree users linked in the README.md.  I searched
> briefly and found:
> https://github.com/advancedtelematic/meta-updater/blob/rocko/classes/image_types_ostree.bbclass
>
> When I was first developing ostree I wrote some code which used libguestfs:
> https://git.gnome.org/browse/gnome-continuous/tree/src/js/libqa.js#n96
> That path has some neat advantages but also disadvantages.
>
> Anyways at a practical level I'd try to find Endless' installer code as it's
> probably the closest match to what you're doing.

Yep, Endless tooling is my starting point on that, I would like to
leverage the existing Debian tooling as much as possible, so I start
looking in this direction first. There's plenty of things available,
building an image from scratch is not the issue.

Thanks for all your answers!

Arnaud