OSTree hosting

[Colin Walters <walters verbum org>]

Hi,

In the past few months (basically since the Montreal conference where a
number of ideas came together) I've been working on a replacement for
jhbuild.  I've been dissatisfied with the way we develop, build, and
deploy GNOME+Linux systems for a long time, and I've tried multiple
things to fix it.  This is my latest, most ambitious system yet.

And to be honest, I think it's really cool =)  I hope GNOME developers
will feel the same, and I want to make a case for why GNOME should host
it.

Effectively my system merges together the good aspects of both jhbuild
and a traditional "package system" like RPM or dpkg (except there are no
"packages" per se).   One of the essential properties of jhbuild that
OSTree maintains is that it *parallel installs* inside an existing
distribution.

What I am using it for, and want to propose for GNOME, is take build a
targeted subset of the FOSS stack between the Linux kernel and
gnome-shell, and use it as a basis for development and deployment.  This
is what *in theory* we have in the jhbuild modulesets today, except
almost no one builds the system dependencies.

The scope of this is limited to GNOME, and particularly to the OS.  I
have no plans to build applications within this system.  How
applications will work is a bit up in the air actually. 

Furthermore, I *definitely* don't want to overlap with what many
"distributions" do by also merging in Java, KDE, cloud tools, multiple
network stacks, etc.  The scope here is and will be tightly focused on
GNOME, and we will continue to make it easy for others to redistribute
our source code and work - I am an expert in both RPM and dpkg for
example, and I know both of them can consume my system. 

So what I need from the GNOME infrastructure is:

 - A server that uses "ostbuild" to rebuild the components of GNOME
   from source control (similar in requirements to jhbuild), that has
   write access to an OSTree repository.  Note that just like jhbuild, the
   build process runs as non-root.  I only need one tiny setuid binary.
   This will need about 30 gigs of disk space, not counting the repository.
   None of the disk space needs backup.

 - Some mechanism for a subset of GNOME accounts to manipulate the build
   server.  This could be a web interface, or it could be just adding
   a subset of the keys to a ~/.ssh/authorized_keys for the build user.

 - Public web server(s) exporting the repository.  The repository size
   *really* varies depending on how much we build.  I think practically
   speaking it's worth keeping it under say 50 gigabytes.   While it
   would be nice for this to be backed up, it's not really necessary.
   If it was lost, we could just recreate it by rebuilding the source.

 - Public web server exporting the build logs for developers to see.  Not
   going to be more than 100MB if that.

For all of this, network bandwidth incoming will be very small.
Network outgoing is more tricky.  Now, the binary repository is designed
to be served out over HTTP, but a given client requesting the full thing
could easily make 20,000 HTTP requests. I plan to have a process where a
"seed" repository can be downloaded via BitTorrent initially.  Another
option is anonymous rsync.  Note though I want to keep the initial target
audience to developers/testers.  We may actually want to restrict access
somehow.

As far as who maintains it - I plan to be primary contact, but I
obviously hope to get more people involved.

Thanks for considering!  More information about the design of the system
is here:

http://git.gnome.org/browse/ostree/tree/README.md