Re: Application bundles in Gnome OS

[Lennart Poettering <lennart poettering net>]

On Mon, 08.10.12 16:49, Alexander Larsson (alexl redhat com) wrote:

Heya,

Sorry for not responding more timely to this mail. I previously have not
been subscribed to the gnome-os ML, but I am now.

> For instance, what part of the core OS do we expect Apps to depend on,
> and what parts would they need to bundle? Obviously Apps need to rely
> on things like a kernel and an Xserver being around, as these can't
> really be bundled. LibGL seems to also be in this category as its
> essentially a driver which can't be bundled. It seems pretty safe to
> also rely on some glibc version and the core X libraries. However, how
> do we go from there? Are apps allowed to rely on bash? coreutils?
> Gtk+? Python? "Robustness" says we should bundle as much as possible,
> whereas efficiency says we should share as much as possible. Where do
> we draw the line?

The way we invisioned this in the app executer stuff we'd like to add to
systemd is we'd offer a number or profiles. Profiles could be something
like "LSB 1.3", "GNOME OS 1.0" or even "NULL". Apps are marked with what
profile they require and can only be executed on OSes that offer the
specific profile. OSes can implement multiple profiles (and usually do),
and apps can require multiple profiles.

A profile is basically a definition of APIs that are supported. The
effect is how much of the host OS is made available to the app container
on execution.  The most drastic profile "NULL" would not make anything
available at all, besides the naked system calls of the kernel. A
profile "GNOME OS 1.0" would provide all APIs that GNOME considers
stable enough to be part of such a profile (and really only those, which
means no libegg or so even though libegg is always installed as part of
GNOME). "LSB 1.3" would provide everything that is standardized by the
LSB.

App designers can then choose what profile to provide their code for. If
they choose "NULL" they of course make their app much more portable than
if they choose "GNOME OS 47.11". However, the backside of the medal is
that in case of NULL they have to ship much more stuff in their app than
they'd have for "GNOME OS 3.0".

Now, what is important here is that these profiles are very high-level,
not finegrained. A profile defines an entire platform, not just small
facets of it. That means app developers can very easily choose what to
develop for and know which APIs they will have available.

Note that nothing that is not part of the chosen profile definition will
be available in the app container. That means stuff lying around in the
host cannot influence the container, unless it is explicitly
whitelested, which is a necessity to make things behave somewhat
consistent among a wide number of distros.

Note that by APIs I actually mean ABIs and bus interfaces alike. For the
latzter this really means that apps cannot access any services on either
the session or the system bus they are not whitelisted for.

> Another problem is that some things are not bundleable. For instance,
> any kind of plugin/extension system break. I.e. you can't expect
> themes or gio-modules like gvfs and dconf to work with a bundled
> Gtk+/glib. And any kind of session-wide service that like gconf, dconf
> or really any dbus using service need to be a unique instance and
> can't be bundled.

In our model we'd basically declare that apps written for "NULL" would
never be able to reuse themes of the host OS. But apps written for
"GNOME 3.0" might, as we'd mount gtk and the plugin subdirs into the app
container.

> The above seems to me to indicate that we need *some* level of
> de-bundling and dependencies, even in an App-bundling system. However,
> we'd like it to be much more coarse-grained than the traditional linux
> packaging system. Basically, any non-extensible library should be
> bundled with the app, and services or extensible libraries should be
> grouped together in larger chunks of ABI backward compatible modules.
> Maybe something like this: (note, these are just dependencies, not
> leaf nodes like apps or the desktop)
> 
> core-1.0 (kernel,systemd,dbus,libc,X11,libGL,pulseaudio,policykit,bash,coreutils,etc)
> glib-2 (glib + gvfs + dconf modules and services)
> gtk-3.2 (cairo/pango/gtk + themes and modules compatible with Gtk 3.2)
> gtk-3.4 (cairo/pango/gtk + themes and modules compatible with Gtk 3.4)
> gstreamer-1.0 (gstreamer + basic set of plugins, you can ship extra
>                plugins in your bundle)

This sounds like the profiles we envisioned, but in my eyes is way too
fine-grained. I believe that the thing is simply that app developers
don't want to develop against their very specific selected set of
versioned deps. They want to write against platforms. Platforms like
"Android 4.0", or "Windows 7 Service Pack 2", or "GNOME OS 1.0", but
nothing beyond that.

> All of these need to be parallel installable or otherwise isolated so
> that multiple abi-incompatible version could be installed at the same
> time.

I'd leave a certain pressure on the folks defining the "GNOME OS 1.0"
profile to only include stuff they consider API stable enough so doing
parallel library installs won#t be necessary for long. That said, if
GNOME ends up breaking a library API, then they should no longer claim
to support apps build for "GNOME OS 1.0", or they should (and this is
much better) indeed provide a parallel-installable version of the lib in
question so that they can continue to support "GNOME OS 1.0" apps.

> There is also the problem what to do with dependencies that have
> unstable ABIs. Take for instance e-d-s, which keeps changing the
> service ABI. Its easy to ship backwards compat client libraries, but
> they won't actually work with a later version of the services. The
> easiest solution to this is to just bundle all the e-d-s using apps
> into one module "desktop" and forbid other apps from depending on it,
> but that might not always work.

APIs not deemed stable should not be available in the profile.

Bus interfaces not deemed stable and not listed in a profile should
probably also not be made available to apps.

> Basically, bundling doesn't completely solve the dependencies issues,
> and we have a lot of work to do to try to balance these issues and
> come up with a coarse-grained dependency system, and then to keep that
> evolving in a backwards compat way.

Yes, it is definitely a difficult balance to find: how much do we
consider stable enough to guarantee to users and how much is too
volatile. Right now it's next to impossible for devs to know that, and
that's why things break all the time. Defining profiles like I proposed
should hopefully be high-level enough to that devs can udnerstand it
while still allowing the platform a certain degree of volatility to
improve.

I'd really like to put the pressure on the folks who define a profile:
they shouldn not add stuff that is not enough ABI stable to the
profile. This should then still allow this to be available to app
developers but it would be implicitly bundled with the app.

Ultimately I think it's a balance that the devs need to find: if they
pick the NULL profile they can some pretty good API compat, simply
because the kernel folks area relatively good at keeping their apis
stable. But then again, you get the least integration in what is around
you. If you opt for more integration you get more volatility and less
stability, but that sounds like an OK deal to me.

Ultimately I believe that games (which need little integration and are
seldom updated) would most likely be attarcted by profiles near "NULL"
or "LSB 1.3" but apps like firefox would be attracted by profiles like
"GNOME OS 3.0" because they update their stuff more frequently anyway
and want the additional integration.

OpenGL is a major point with ABIs here. If an app wants to use OpenGL we
must mount this lib in from the host (since the nvidia binary drivers
need that). OpenGL also pulls in a specific C++ API. This probably boils
down to that OpenGL apps will necessarily be unable to be written with a
"NULL" profile, unless they stick to software-only rendering.

Anyway, so much about my ideas on this.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.