Thoughts on a new network repository mode: subtrees

[Colin Walters <walters verbum org>]

The current "exploded" archive-z2 repository mode has a lot of tradeoffs
- I think it's proven to be pretty good for a "continuous delivery"
mode, but not everyone only cares about CD of the most recent version.

As discussed here:
https://bugzilla.gnome.org/show_bug.cgi?id=721799

It's quite inefficient when going from major version -> major version. 
Concretely, I tried going from a Fedora 21 Docker host tree to rawhide,
and it was ~11000 HTTP requests.  It did work, but yes, very slow.

The static deltas bug above I think will work quite well for OS creators
who ship infrequently, with large changes in betweeen versions.

But one thing I was thinking about is going back to something very
similar to the package model, where you still have a tree (i.e. precise
binaries), but on the server side, we effectively have each package
version as a tarball.  

There would be a lot less deduplication on the server side, but an
interesting aspect of this is that it becomes a *lot* less expensive to
switch between arbitrary versions - you don't have the
death-by-http-request overhead.

Yes, you download effectively the whole OS again, but people have lived
with that.

If we kept the "subtree" repository model generic, it would still allow
for the build side to deduplicate as it wants.  Although in practice,
people do this on the package level anyways, by splitting out large game
data into noarch subpackages.