Re: Scalability of OSTree

["Jasper St. Pierre" <jstpierre mecheye net>]

Interesting. There are other issues with the summary file for us (all
branches being public), and we expect to scale significantly in the
future, so having an ever-growing file the client has to download
isn't particularly appealing.

Is there anything the summary file does that couldn't be better
approached through per-branch files in the repo itself?

On Thu, Dec 3, 2015 at 3:06 PM, Alexander Larsson <alexl redhat com> wrote:
> On Thu, 2015-12-03 at 10:53 -0800, Jasper St. Pierre wrote:
> > Hey,
> >
> > Here at Endless, we're looking at creating a binary distribution
> > system not for operating systems or applications, but for application
> > content and media. OSTree's binary distribution mechanism parts solve
> > a lot of the issues of upgrading content, and we're looking at
> > reusing
> > it much like xdg-app reused that part as well.
> >
> > Similar to xdg-app, we're imagining that one master repo would hold
> > many refs, which would be our content sources. Users would have on
> > average 100 - 1000 of these branches active at once. Our servers
> > would
> > host a large number of these branches as well. To give an example of
> > scale, if we deployed it today, we would have roughly ~300 branches
> > on
> > our server.
> >
> > We're very conscious of bandwidth and other resources here, and it's
> > a
> > bit unfortunate that OSTree is extremely network chatty. Static
> > deltas
> > do help cut down on the number of object requests, but there's still
> > a
> > lot of overhead in simply resolving a ref.
> >
> > An especially major paint point would be the summary file -- having
> > every bit of content in one repo would mean that the summary file
> > could grow to be tens of megabytes, which isn't acceptable to us.
>
> That sounds like a lot. The gnome-sdk summary file is 316K:
>   http://sdk.gnome.org/repo/summary
>
> It contains 2935 branches.
>
> That said, I think the way ostree handles the summary file could use
> some work. We always download the summary on every pull, even if it was
> just recently downloaded. For instance, if you're doing an update of
> all apps from a remote with xdg-app it will download the summary file
> once per app. I think we should have a local cache of the summary file
> for each remote, and then use an etag to verify when it didn't change
> (and maybe even have some local timestamp so that we don't even make a
> request if it was downloaded very recently).



-- 
  Jasper