Re: Application bundles in Gnome OS

[Alexander Larsson <alexl redhat com>]

On Tue, 2012-09-18 at 22:21 -0400, Colin Walters wrote:
> On Thu, 2012-09-13 at 19:01 +0200, Alexander Larsson wrote:
> 
> > https://github.com/alexlarsson/bundler
> > 
> > It basically just loopback mounts a squashfs file in a private mount
> > namespace in a hardcoded place and execs a hardcoded binary name from
> > it. There is no desktop integration and no other flexibility, although
> > things like that *could* be introduced by a separate daemon that
> > extracts metadata from "installed" bundles.
> 
> Right.
> 
> You could go even more minimal - an application is literally just an
> executable installed in /opt/globally-unique-name.

Not sure what you mean here? Thats the result e.g. bundler gives you, a
single ELF file that you can run. It just makes it easier to construct
such an executable from existing codebases. How is what you mean
different?

> It's not clear to me that it's a good idea to mandate use squashfs or
> something like that - yes, you save disk space, but at a runtime cost.
> Demand paging becomes more complex; reading in that next page may
> involve a substantial amount of decompression.

Squashfs need not be compressed, thats selectable at image-build time. 

However, gzip (the default compression mode in squashfs) is pretty fast,
and given modern CPU speeds vs disk speeds it's quite likely that
reading a smaller file and uncompressing it is faster than reading the
uncompressed file. 

And if its not, we could use something like snappy[1] which definately
is fast enough for this.

Doing unbundling of the installed apps is still an interesting idea
though, as we could then unbundle to sha-1 names in a single directory +
hardlinks in the app, which gives us instant memory sharing.

> There is one runtime feature that I think should be mandated - a cache
> directory writable (and executable) by the application, separate from
> the home directory.

This is somewhat problematic in that you can have multiple instances of
a single app running, although I guess we could punt this to the app to
handle (i.e. its no different than a non-bundled version).

> This route is the total opposite from what Ubuntu is looking at with
> their application upload process.  But I think it's most interesting to
> look at what features are needed *fundamentally*, and what is
> conceptually sugar on top.

One question that comes to mind is, do we fundamentally need sandboxing?
That is the single feature that will be hardest to implement and affect
how people bundle apps most. It also means most apps won't work as-is.

Maybe sandboxing could be made optional for untrusted (non-signed) apps.
That way it can be introduced while still allowing current linux apps to
work as-is, if packaged by some trusted source.

> Core data files like translations could just as easily be embedded ELF
> resources (as you know being the author of GResource, but worth
> mentioning for others).

bundlers puts the whole FS inside an ELF resource, which is essentially
the same thing, except the already existing tools like the libc
translation support works as-is without needing to rewrite the world.

> One final thing - I'd love if whatever bits we have on the application
> source side are focused around git repositories as input.  The build
> system should just Do The Right Thing when presented with one.

Git repositories do not have enought metadata to build executable
bundles though. Using something like jhbuild or similar is a great way
to get *something* that compiles and runs. But at the end you will end
up with a single shared prefix that has all the build requirements and
all the devel/example/tools files for everything you built. But what you
need to bundle is the minimal set of runtime dependencies that your app
needs.

So, in addition to pointing to the git repo and knowing how to build and
install it into a prefix we need metadata for:

* Which other src-modules are needed to build this src-module
* Which files are needed at runtime.
* Which other binary-module-subsets are needed at runtime.
* What scriptlets do we need to run once we've put the required runtime
  files in the bundle.

Additionally, it seems like when building a bundle like this it would be
nice if you were able to re-use pre-built modules for most dependencies.
This means you will end up with identical binaries in many bundles,
which lets us do memory-sharing, but it also lets us pool resources (for
development and bundling) of these things.

This seems a *lot* like what current packaging systems do. In fact, you
could easily imagine using specfiles and rpms built to a separate
prefix, in a separate rpmdb to handle building such bundles.

[1] http://code.google.com/p/snappy/