Re: OSTree hosting

[Jeff Schroeder <jeffschroeder computer org>]

On Fri, Jan 27, 2012 at 11:09 AM, Colin Walters <walters verbum org> wrote:
> Hi,
>
> In the past few months (basically since the Montreal conference where a
> number of ideas came together) I've been working on a replacement for
> jhbuild.  I've been dissatisfied with the way we develop, build, and
> deploy GNOME+Linux systems for a long time, and I've tried multiple
> things to fix it.  This is my latest, most ambitious system yet.
>
> And to be honest, I think it's really cool =)  I hope GNOME developers
> will feel the same, and I want to make a case for why GNOME should host
> it.
>
> Effectively my system merges together the good aspects of both jhbuild
> and a traditional "package system" like RPM or dpkg (except there are no
> "packages" per se).   One of the essential properties of jhbuild that
> OSTree maintains is that it *parallel installs* inside an existing
> distribution.
>
> What I am using it for, and want to propose for GNOME, is take build a
> targeted subset of the FOSS stack between the Linux kernel and
> gnome-shell, and use it as a basis for development and deployment.  This
> is what *in theory* we have in the jhbuild modulesets today, except
> almost no one builds the system dependencies.
>
> The scope of this is limited to GNOME, and particularly to the OS.  I
> have no plans to build applications within this system.  How
> applications will work is a bit up in the air actually.
>
> Furthermore, I *definitely* don't want to overlap with what many
> "distributions" do by also merging in Java, KDE, cloud tools, multiple
> network stacks, etc.  The scope here is and will be tightly focused on
> GNOME, and we will continue to make it easy for others to redistribute
> our source code and work - I am an expert in both RPM and dpkg for
> example, and I know both of them can consume my system.
>
> So what I need from the GNOME infrastructure is:
>
>  - A server that uses "ostbuild" to rebuild the components of GNOME
>   from source control (similar in requirements to jhbuild), that has
>   write access to an OSTree repository.  Note that just like jhbuild, the
>   build process runs as non-root.  I only need one tiny setuid binary.
>   This will need about 30 gigs of disk space, not counting the repository.
>   None of the disk space needs backup.
>
>  - Some mechanism for a subset of GNOME accounts to manipulate the build
>   server.  This could be a web interface, or it could be just adding
>   a subset of the keys to a ~/.ssh/authorized_keys for the build user.
>
>  - Public web server(s) exporting the repository.  The repository size
>   *really* varies depending on how much we build.  I think practically
>   speaking it's worth keeping it under say 50 gigabytes.   While it
>   would be nice for this to be backed up, it's not really necessary.
>   If it was lost, we could just recreate it by rebuilding the source.
>
>  - Public web server exporting the build logs for developers to see.  Not
>   going to be more than 100MB if that.
>
> For all of this, network bandwidth incoming will be very small.
> Network outgoing is more tricky.  Now, the binary repository is designed
> to be served out over HTTP, but a given client requesting the full thing
> could easily make 20,000 HTTP requests. I plan to have a process where a
> "seed" repository can be downloaded via BitTorrent initially.  Another
> option is anonymous rsync.  Note though I want to keep the initial target
> audience to developers/testers.  We may actually want to restrict access
> somehow.
>
> As far as who maintains it - I plan to be primary contact, but I
> obviously hope to get more people involved.
>
> Thanks for considering!  More information about the design of the system
> is here:
>
> http://git.gnome.org/browse/ostree/tree/README.md

Perhaps naiive question, "Could many of these problems be solved by a
GNOME hosted version of the OpenSUSE build service?"

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com