Re: New gnome-sdk images with gnome-builder app bundle

[Alexander Larsson <alexl redhat com>]

On ons, 2015-01-14 at 12:36 +0800, Cosimo Cecchi wrote:
> Hey Alex,
>
>
> On Tue, Jan 13, 2015 at 11:49 PM, Alexander Larsson <alexl redhat com>
> wrote:
>         2) You want the app to "come with" the platform, so that you
>         don't have
>         to download it separately. I don't think we really need to
>         "bundle" the
>         platform in the strict meaning (put the files in it), however
>         we could
>         perhaps ship the platform in the same ostree repo, or somehow
>         point to a
>         repo that has it. Then the runtime reference could be
>         automatically
>         downloaded by the installer.
>
>
> When apps are provided by the same source that also built the OS with
> all the runtimes, this comes natural to do. I think one of the
> greatest advantages of this system is that it allows third-party
> developers to easily ship their own version of platform libraries that
> would traditionally be exclusively provided by the OS.
> So yes, I think downloading an extra platform or parts of it will be a
> common operation. See more thoughts about this below.

I would be hesitant in general for too many people to write runtimes.
Implementing a runtime is essentially writing an maintaining an
operating system, with extra constraints that it has to work with any
host system.

In fact, a runtime by itself has some very concrete requirements from
the host OS. For instance, a runtime could ship the client libraries for
dbus services like gvfs or dconf, but they rely on it existing in the
host (with a ABI-binary-compatible version). Things are even more
complicated when it comes to things like OpenGL libraries which have
ties to specific kernel drivers (not to mention 3rd party proprietary
drivers).

So, basically, a runtime is more than just the set of ELF abis that it
guarantees. It also says something about the host system.

For this reason i think the set of runtimes that are in wide use will be
limited. It is also the reason why i think its likely that distros will
want to build some of the runtimes themselves, so they can ensure the
right dependencies here. And for this reason I think its not a great
idea for an app author to do their own runtime. They could of course
decide to *ship* someone elses runtime themselves (although they are
then responsible for security updates, etc). 

>         > - moreover, if org.wxwidgets.Platform depends on some GNOME
>         platform
>         > libraries, do you specify that as a dependency between
>         runtimes (e.g.
>         > requires org.gnome.Platform) or is it left to the OSTree
>         layer to
>         > deduplicate the dependencies if org.wxwidgets.Platform and
>         > org.gnome.Platform have been built in the same repo/initial
>         buildroot?
>         
>         We definately don't want to be in the business of re-inventing
>         a package
>         system where runtime depends on each other with versioning and
>         whatnot.
>         If someone wants to make a platform that has parts of
>         org.gnome.Platform
>         that is up to them. They could chose to use all or parts of
>         org.gnome.Platform, which will automatically cause ostree to
>         dedup the
>         two (but they'd have to do their own security updates, etc).
>         
>         In fact, we should perhaps even join forces with other orgs
>         (kde,
>         freedesktop) to share the base of org.gnome.Platform.
>
>
> I agree that a base all runtimes can assume is a good idea.

I don't think *all* runtime need to use it, but it is good to *have* one
so that work on this can be shared by anyone.

> I think one basic question is: can a platform require another
> platform, or is the only assumption possible for a runtime that the
> shared base will be there?

No, there can be only a single /usr, which is the runtime. At least at
the level of the deployed data. Of course, you can layer something above
this so that you could create platforms in some kind of hierarchical
way. I don't think that is a good idea though, as each of these would
then be its own chain of development with security updates, syncing with
the host os, etc.

I think rather that we should make widely used base runtimes contain the
most basic stuff that everyone uses and that interacts with the host in
ways that make demands of the host os. Then we put anything above this
bundled in the apps. 

That said, the thing we bundle in apps could itself be built in such a
way that it is easy to reuse for separate apps that wishes to bundle it.

> Let's assume the answer is no, and take the WxWidgets as an example;
> surely WxWidgets will need things like GLib and GTK2 to build, with
> all of their dependencies. It might just be a matter of using the
> right tooling, but I believe anything that requires reproducible,
> automated builds will want these dependencies to be encoded somehow in
> a WxWidgets "recipe" of some sort. Whoever builds the library then
> should be able to pull those dependencies individually from the
> underlying OSTree storage to assemble the buildroot. Is this provided
> by the RPM layer in your implementation?

Yeah, the way I see this is that wxwidgets would chose a runtime they
want to use, which is either the gnome one or a more limited freedesktop
one. Then they create a set of rpms built to install in /self and depend
on the /usr from the chosen runtime. An app can then easily install
these rpms + its own app files and dependencies when they create the app
bundle (which also uses the above specified runtime).

This looks inefficient, but ostree will make things share space in the
repo (and even during download).

> If you assume the answer is yes, then e.g. WxWidgets could just
> require the GNOME platform to be built and implicitly get the
> dependencies; the application could just require the WxWidgets
> platform which is then provided in the repo.

This is much more complex in reality. The wxwidget platform would have
to depend on a specific platform version, and you may need multiple
"addon" platforms that depend on various versions (and each other). And
there is no simple way to combine these as they may overlap in weird
ways and require post-install triggers like update-desktop-database.
Basically you're re-creating RPM and the things we're trying to get away
from.

> This is where things get a bit fuzzy for me; where do we draw the line
> between a platform - a set of libraries/versions that provide
> functionality the application can rely on and build against - and just
> a library that the application needs to bundle using a system similar
> to the above?

Yes, there is no "obvious" line here. We have to decide where to best
draw the line, and once we've done so, anyone who choses to use that
runtime will know what they need to bundle.

Here are some of the things we have to consider:

* Some things by necessity need to be in the runtime because they
interact heavily with the host OS and may need tweaking if the host
environment changes. So, things like wayland, (k)dbus, X, OpenGL, audio,
video, devices, dns resolving, proxies, core dbus desktop services, etc.

* Things that are very security sensitive, like image libs, crypto libs,
etc, are very nice to have in the runtime, because then they share
security updates with all other apps (that may not be as good at doing
updates).

* Some things are quite large, like the locale data and icon sets, which
is nice to have guarantees of sharing between apps.

* Some things are hard to build (glibc and other lowlevel stuff), others
are just painful (webkit). Its very nice for app authors if they can
just rely on this stuff and not have to build and bundle it themselves.

* On the other hand, we want the runtime to be small. Both for
storage/download reasons, and because we want to limit the
maintainance/update load for it. Its very easy to feature creep it to be
a full distro which is not where we want to go.

* Anything we add to the runtime has the potential to conflict with
something bundled by the application (if it needs a different version). 

I've currently made the platform pretty minimal. For instance, it
doesn't even have python or perl. I think we need to carefully consider
anything more we add to it.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl redhat com            alexander larsson gmail com 
He's an unconventional sweet-toothed librarian gone bad. She's a mentally 
unstable tomboy doctor with a birthmark shaped like Liberty's torch. They 
fight crime!