Re: apt2ostree - build ostree images of apt based distributions

[Daniel Drake <drake endlessm com>]

On Thu, Dec 13, 2018 at 3:00 AM Felix Krull <f_krull gmx de> wrote:
> On Wed, 12 Dec 2018 at 18:57 Will Manley <will williammanley net> wrote:
> > On Wed, 12 Dec 2018, at 05:31, Daniel Drake wrote:
> > > On Tue, Dec 11, 2018 at 4:28 AM Colin Walters <walters verbum org> wrote:
> > > > Cool!  Would love if the deb-ostree-maintainers had a chance to look at this - there's a specific 
> > > > comparison here: https://github.com/stb-tester/apt2ostree#endless-ostree-builder-eob
> > >
> > > I imagine the deb-ostree-builder is lagging behind what we use
> > > internally, sorry about that.
> > > The comparison seems pretty accurate at a high level.
> > >
> > > However, in our updated internal version, we have stopped using
> > > debootstrap to set up the base filesystem tree. We got fed up of
> > > debootstrap acting differently from the real apt & dpkg
> > > implementations. We tried to fix it up, we tried to use multistrap too
> > > (same issue there), it just got too painful and instead of continually
> > > trying to adhere closer (and keep up with) apt, we decided to just use
> > > apt itself.
> > >
> > > If apt2ostree does not actually use apt for dep resolution, package
> > > installation etc, then I suspect it will ultimately face the same
> > > headaches.
> >
> > Thanks for the heads-up.  Did you find the issues were more with apt dependency resolution and package 
> > selection, or more on the installation/dpkg side or both?
>
>
> I've been thinking about the topic a bit, and I've come to a few conclusions:
>
> * There's no reason not to use apt for dependency resolution. As I understand, debootstrap only doesn't so 
> it doesn't depend on Debian-specific things.

Yeah, we've hit a bunch of problems with faced with debootstrap
dependency resolution.

Such as this one: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=878961

The one that really drove us away was the case where you have multiple
versions of the same binary package available in different apt repo
components (or different apt repos). Here, apt has well defined
behaviour - it'll pick the one with the highest version number, and we
needed this behaviour for a somewhat complex use case. debootstrap
doesn't do that (I can't recall if it picks the first one it finds or
if it is basically random).

> * You probably can't run any preinst scripts. This is because you need either a (partial) filesystem tree 
> (if it's a shell script) or binary emulation (if it's compiled, as is e.g. the case for the dash package). 
> In general, it seems to me that running package code during the unpack phase is probably out of the 
> question.

And that's another big problem. Debian packages are written with the
expectation that preinst scripts get ran, and if you're not running
them then you are going to run into strange inconsistencies and/or
outright failure down the line.

I think multistrap does allow you to run preinst scripts, but it
actually runs the preinst scripts after installation, ignoring the
"pre", which is totally not what's supposed to happen, and Debian
preinst scripts are widely written with the expectation that they run
before install (as is documented), so that part of multistrap is
rather broken.

If it's really not possible to comply with some aspects of standard
Debian package behaviour (such as preinst scripts) in a deb-to-ostree
type system then that's a pretty major concern in my eyes. Maybe it's
worth it for some use cases where you have tight control over the
packages and can accept the extra hassle, especially if it's a
trade-off that gives you certain gains too, but you'd want to flag
this up in comparisons and be cautious when presenting it as a generic
deb-to-ostree type tool.

Daniel