Scalability of OSTree

["Jasper St. Pierre" <jstpierre mecheye net>]

Hey,

Here at Endless, we're looking at creating a binary distribution
system not for operating systems or applications, but for application
content and media. OSTree's binary distribution mechanism parts solve
a lot of the issues of upgrading content, and we're looking at reusing
it much like xdg-app reused that part as well.

Similar to xdg-app, we're imagining that one master repo would hold
many refs, which would be our content sources. Users would have on
average 100 - 1000 of these branches active at once. Our servers would
host a large number of these branches as well. To give an example of
scale, if we deployed it today, we would have roughly ~300 branches on
our server.

We're very conscious of bandwidth and other resources here, and it's a
bit unfortunate that OSTree is extremely network chatty. Static deltas
do help cut down on the number of object requests, but there's still a
lot of overhead in simply resolving a ref.

An especially major paint point would be the summary file -- having
every bit of content in one repo would mean that the summary file
could grow to be tens of megabytes, which isn't acceptable to us.

There are also other requirements we would want as well, like the
ability to mirror these systems to CDNs. There has been talk about
using it for store-bought applications as well, which means that we
would want encrypted objects.

We would be fine putting in engineering effort towards building things
like smarter servers so clients can ask for 200 branches at once, the
server would resolve it to objects they would have to download, and
send URLs for those out.

Would the OSTree project want something like this, or would it be
better for us to build a custom distribution system from scratch?

(Also, would it make sense to split up the binary distribution parts
OSTree from the Linux deployment parts? Perhaps a more focused, robust
binary distribution component would thrive independently a lot more
than something attached to Linux deployment. Or do you think the
requirements are so specific for each use case that having it as a
generic component doesn't quite make sense)

-- 
  Jasper