Re: Repo scalability issues and solutions

[Jonathan Dieter <jdieter gmail com>]

On Wed, 2020-08-26 at 09:26 +0200, Alexander Larsson wrote:
> I didn't know about zchunk, but I took a look. It looks like a fine
> solution for some things but I don't think it is a good fit here.
>
> First of all, ostree is already a widely deployed system that has a
> certain design. The changes I talk about are minor restructurings of
> the details of that, not completely dropping in some external library
> that does things in a different way.
>
> Secondly, zchunk fundamentally depends on ranged downloads, and that
> is
> not something we currently require of our http servers, nor is it a
> good idea as it can interfere with things like CDNs.
>
> Third, the rolling checksum chunking that zchunk does is similar to
> what we already do in the fallback case for ostree static deltas. But
> those perform significantly worse than the primary delta approach we
> use, which is bsdiff based. So I don't think zchunk will perform as
> well as what we currently use.

That's fair enough.  If ranged downloads aren't acceptable then zchunk
is definitely out the window (leaving apart the other issues).

Thanks for taking a look.

Jonathan