gnome-sdk update and TODOs

[Alexander Larsson <alexl redhat com>]

Since my last post on gnome-sdk I've implemented a new approach of
building the base images. It is based on a refined version of the yocto
bases that gnome-continuos uses, slimmed down a bit to only be the core
libraries and binaries (i.e. nothing glib or desktop related at all).
Then all the gnome-maintained and desktop related stuff is build using
rpm, rather than using gnome-continuos. This allows us a bit more
control of exactly what ends up in the final images and it also makes it
easier to (optionally) use rpms to build your apps.

I put up an image of an ostree repository with the sdk/platform images and
a GEdit app at:

http://people.redhat.com/~alexl/gnome-sdk-repo-2014-11-28.tar

To test, first build gnome-sdk:
  https://github.com/alexlarsson/gnome-sdk

Make sure you sudo make install, because gnome-sdk-helper needs to be setuid.

Then untar the repo somewhere and run something like:

 gnome-sdk-repo add-remote test-repo file:///dir/to/untarred/repository
 gnome-sdk-repo install-runtime test-repo org.gnome.Platform 3.14
 gnome-sdk-repo install test-repo org.gnome.GEdit
 gnome-sdk-run org.gnome.GEdit gedit

You can also install the sdk with:

 gnome-sdk-repo install-runtime test-repo org.gnome.Sdk 3.14

You can also install the /var images containing the rpm DBs for 
Platform/Sdk. These are useful with the gnome-sdk-build -v option 
as you then get an initialized rpm db that you can use to build/install
further packages:

 gnome-sdk-repo install-runtime test-repo org.gnome.SdkVar 3.14
 gnome-sdk-repo install-runtime test-repo org.gnome.PlatformVar 3.14

If you want to build the images yourself, then clone:
  https://github.com/alexlarsson/gnome-sdk-images

And just type "make". This will download the yocto snapshot from

  https://github.com/alexlarsson/gnome-continuous-yocto/tree/gnomeostree-3.14-dizzy-platform

and the sources needed from ftp.gnome.org. It will then build the
yocto base and the rpms for the gnome parts and create tarballs with
the final images.

Then you can type "make commit", which will create an ostree repository
with these.

All this takes a while, so If you just want to play i recommend using
the prebuilt repo above.

This platform is a very minimal base to start from. It is enough to
run basic stuff, but its has not really tackled the hard problems.
First of all, it doesn't really do any sandboxing at all, as that is
just impossible without underlying changes like kdbus and app
changes. But I want to start with trying to make it just work for
current non-sandboxed apps, and there are a lot to fix just for that.

Here is a list of things that I will start to look at now that we have
a base to work from:

* X11 connection

 X uses both a normal and an abstract pathname for its unix domain
 socket.  This is non-ideal, because the abstract pathname is global
 (not namespaced) which means all X servers listening on the system
 will be accessible to the app, not only the one we bind into the
 container. Not sure what the best approach is here, probably just
 don't have the system Xserver listen to abstract pathnames.

 In general X11 is completely insecure and needs a shared IPC
 namespace for Xshm to work, so it is inherently not sandboxable.
 Still, I think we should let the app access as little as possible

* DConf

 How do we expect things like the system default db to work if we
 can't access the host filesystem? Should we mount it in? Should we
 request fds for it from the daemon?

 Also, in a sandboxed version with no access to $HOME, how do we read
 configs? Per-app config db? How do you then manage that globally from
 the host?

* GVfs

This mostly works, but I had to hack it to make the remote volume
 monitors to work, because that works by reading a bunch of .monitor
 files from the system dir to know what dbus services to use. This
 should all be accessible from the main gvfs daemon via dbus.

* OpenGL

 There is no opengl yet in the base image, and making this work will
 be a lot of work, since mesa uses udev and dri devices, relies on
 particular kernel/Xorg drivers, etc. It also needs to be replacable
 easily as the way you install a different gl driver is to replace the
 entire GL library.

 Mesa is also pretty unsafe in terms of sandboxing for anything before
 DRI3, as DRI2 buffer handles are global and guessable. Dunno about
 e.g. nvidia.

 This will require a lot of ad-hoc work, and probably some setup to
 replace part of a runtime with another sub-runtime.

* Audio

 There is no audio support atm. Neither alsa libs, not device nodes.
 I think the best approach is to use pulseaudio, but this needs
 careful consideration in terms of ABI/IPC compat, performance,
 latency, etc.

* DNS Resolver

 We currently don't provide a resolv.conf, so dns just doesn't
 work. But fixing this is hard. We can copy one at startup, but that
 would not be updated as we change between network, etc. I know
 lennart is working on a dbus based nss backend for dns resolving, and
 i think this is the best long term approach. Maybe we can do some
 hackery to make it mostly work for now though? Maybe a session dbus
 service?

* Network Manager

 A lot of apps will want to rely on info like whether we're online
 from network manager. We need to figure out what is needed and how we
 can safely supply it to the app.

* HTTP proxy config

 We need the app to follow the proxy configuration from the host,
 without having each app do it completely by itself, which implies a
 session service.  Need to look into how this would work.

* Timezone

 We need to get current timezone info into the app. Maybe we also want
 updated timezone info from the host?

* Accessibility

 I don't know the details here, but how do we make contained apps a11y
 enabled, talking to the session a11y services.

* Printing

 How do we make it possible for contained apps to print? We can't have
 each app implement print configuration reading, etc. Somehow this
 needs to involve a cupsd in the session or something.

* Color calibration

 No clue about colord, etc, needs research.

* Avahi

 Can we just ship the avahi client libraries in the platform and
 expect this to work?

* Device access

 Some apps need specific device access. For instance, many games
 directly access input devices, and some apps need usb access to
 particular devices. How do we handle this?  Also, udev has no ABI
 stability guarantees for its runtime db, so we can't really use
 libudev much inside the app bundle.

* Bundle more in the platform?

 Right now the platform is very minimal. For instance, it doesn't have
 python. Adding things to it makes it easier for app authors, but it
 increases the maintainance burden on gnome, as well as possibly
 making things complicated if applications wants to bundle the same
 dependencies but with different versions.

 This needs careful thought about where we draw the line.

* ostree clone performance

 The ostree repository format is pretty inefficient for the first
 download as you have to do a http request per file, we want something
 like the pack files that git has.

 I think the atomic project is looking at this, as they have similar
 issues.

* ostree local repo format

 The ostree local repository format stores uids as real file
 uids. This means that you have to be root to clone a repo with
 root-owned files in it.  This makes it hard to do user-installed
 apps.

* Build more apps

 Just to verify things we need to build more apps and test them

* Localization

 Translations and locale data are pretty large. Just installing all
 the base glibc locale data is 80 megs of compressed data.  Just the
 gtk3 translations for all languages are 7 megs.  Maybe we can somehow
 make this separated out and optional.