Re: New gnome-sdk images with gnome-builder app bundle

[Alexander Larsson <alexl redhat com>]

On tis, 2015-01-13 at 12:29 +0800, Cosimo Cecchi wrote:
> Hey Alex,
>
>
> This sounds fantastic! I have been spending some time in the past and
> more recently thinking about the same problem, which we have solved in
> Endless with a lot of hacks and without sandboxing, and what you have
> is elegant and I really believe the right way forward for a proper
> upstream solution to the application story.
>
>
> A few questions off the top of my head:
> - how is the OSTree repo that contains all the runtimes and 
> applications built? Are doing this using Fedora/RPM packages or Yocto?

Well, the repo itself it just created with ostree like so:
 ostree  init --mode=archive-z2 --repo=repository
 ostree commit --repo=repository --branch=runtime/org.gnome.Platform/x86_64/3.16 --no-xattrs commit

After commiting what i want here i just copy the result to the
webserver. However, the *contents* of the runtime is more complex.

The core is based on yocto. I have a branch based on the
gnome-continuous stuff that is tweaked to match better what we need
(i.e. no hw support, etc). It is available here:
https://github.com/alexlarsson/gnome-continuous-yocto/tree/gnomeostree-3.14-dizzy-platform
In particular the we build the gnomeos-contents-sdk and
gnomeos-contents-platform targets.

This core is completely devoid of any highlevel dbus/gnome/freedesktop
stuff, it just has the core libraries and binaries. This is nice because
we get a common core that can be built anywhere and that has an upstream
doing security releases.

Then on top of that there is a layer that contains all the "desktop"
stuff, including all the gnome.org and freedesktop.org stuff. This is
the things where we want complete control and need to do special stuff
for the sandboxed usecase. These are built separately as rpms, and much
of it is based on specfiles from fedora (although these are pretty
vanilla as fedora don't change upstream much).

The rpms are available here:
https://github.com/alexlarsson/gnome-sdk-images

This repo has the specfiles, and a makefile that is set up to
autogenerate make dependencies from the specfile to download the sources
and build the rpms. It also has makefile rules to download and build the
yocto base and to create a ostree repo from the result.

In fact, you should need only to check this out and type
"make; make commit" which will give you a "repository" directory with
the runtimes.

The apps I've created so far are much more ad-hoc. I've just taken pure
fedora srpms that are needed for the apps and built them in "xdg-app
build" which automatically sets the right prefix and whatnot (some srpms
needed minor tweaks). Then i just used ostree commit to create the app
repo. This could obviously use better some tooling.

> - I assume files are deduplicated in the repo storage, so that e.g. if
> a repo has two runtimes with files in common, they are shared at the
> filesystem level?

Yeah, they are stored as content-addressed sha256 names which are then
hardlinked into the checkouts.

> - in a world where this system is the standard, how do you envision
> runtimes to be built, distributed and installed? I can see things
> going either in the direction of distributions providing runtimes that
> are compatible with a specific standard (e.g. GNOME 3.16, KDE 4.5,
> etc), or each "software foundry" providing their own directly, or a
> mix of both...

I would be lying if i said i know exactly how this will work out, but I
believe that gnome will produce a runtime that is a) available as a
binary, and b) easily rebuildable by distros. And then distros will
probably ship "some" supportable runtimes that they rebuild (so they can
fix security issues, make sure things like opengl drivers match whats in
the runtime, etc). 

I also expect a few other major orgs like kde and maybe LSB to make
runtimes. Possibly also some distros (based on the disto packages).

> - related to above, is it possible in the system design to download a
> runtime on-demand if the application requires it? I can imagine this
> being useful for things like Flash or non-free codecs/drivers

This is a pretty high-level operation, It could easily be added if we
think it makes sense. It means however, that the app will have to
somehow reference a repository where the runtime is (at least if its a
non-standard one).

> - how do you package an application bundle? More in detail, imagine I
> want to build a net.sourceforce.audacity application that requires an
> org.wxwidgets.Platform. I think I would like a system where
> org.wxwidgets.Platform can either be provided with the bundle itself
> or be available in the OS.

Well, first of all i'm not sure wxwidgets is broad enough to be its own
platform. You don't want some wxwidgets guy to maintain an entire OS
just to package a UI library. I think rather you'd use a more generic
platform (maybe a freedesktop one with just X+dbus) and then you bundle
wxwidgets inside your app. So there are two issues here:
1) Many apps will probably want to bundle wxwidgets, so we should make
this easy with tooling. Also, ideally such tooling should result in the
multiple apps using binary identical library files so that we get
hardlink sharing.
2) You want the app to "come with" the platform, so that you don't have
to download it separately. I don't think we really need to "bundle" the
platform in the strict meaning (put the files in it), however we could
perhaps ship the platform in the same ostree repo, or somehow point to a
repo that has it. Then the runtime reference could be automatically
downloaded by the installer.

> - moreover, if org.wxwidgets.Platform depends on some GNOME platform
> libraries, do you specify that as a dependency between runtimes (e.g.
> requires org.gnome.Platform) or is it left to the OSTree layer to
> deduplicate the dependencies if org.wxwidgets.Platform and
> org.gnome.Platform have been built in the same repo/initial buildroot?

We definately don't want to be in the business of re-inventing a package
system where runtime depends on each other with versioning and whatnot.
If someone wants to make a platform that has parts of org.gnome.Platform
that is up to them. They could chose to use all or parts of
org.gnome.Platform, which will automatically cause ostree to dedup the
two (but they'd have to do their own security updates, etc).

In fact, we should perhaps even join forces with other orgs (kde,
freedesktop) to share the base of org.gnome.Platform.

> - finally, do you have a wiki page online or something where I can
> find more information on the design of the system?

I've posted some to gnome-os-list and there is some text here:
https://wiki.gnome.org/action/info/Projects/SandboxedApps

Also there are the old stuffs which this is based on:
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html
https://docs.google.com/document/d/1QTgxakyUVFMkvr-xFY2Xg9lYjcJLd6kPTl3Ij5_dL7Q

But in general, there is not a lot of docs yet.

> Thanks! I will be attending the DX hackfest in Cambridge at the end of
> the month and I'm very much looking forward to chat about this.

I'll be there too.


-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl redhat com            alexander larsson gmail com 
He's a lounge-singing amnesiac grifter in drag. She's a bloodthirsty 
kleptomaniac angel with someone else's memories. They fight crime!