Re: Sizes data in flatpaks

[Alexander Larsson <alexl redhat com>]

On Fri, 2020-10-09 at 07:12 -0600, Dan Nicholson wrote:
> Anyways, I wanted to see how this would affect flatpak, so I wrote a
> script (attached) to fetch refs then recommit them with the sizes
> metadata. Here are some results for some popular refs.
>
> Ref                                              Objects  Current
> With Sizes    Change
> ---------------------------------------------  ---------  ---------
> ------------  --------
> runtime/org.freedesktop.Platform/x86_64/19.08      11266  2.6 KiB
> 456.4 KiB     +172.1x
> runtime/org.gnome.Platform/x86_64/3.38             20092  2.4 KiB
> 810.2 KiB     +336.2x
> app/org.gimp.GIMP/x86_64/stable                    10297  1.4 KiB
> 415.2 KiB     +287.0x
> app/org.mozilla.firefox/x86_64/stable              20220  1.5 KiB
> 814.4 KiB     +526.8x
> app/com.spotify.Client/x86_64/stable               21214  1.8 KiB
> 854.2 KiB     +481.8x
>
> It definitely makes a big difference, although none of these are over
> 1MB like our OS commits are.

These are not near the 10MB metadata size limit, but they are still
quite large. For instance, it doesn't seem useful (too slow) to
download these before starting the actual flatpak download operations,
so it will not help produce more useful initial estimates for the
downloads in the flatpak cli prompt. That particular problem is imho a
larger problem than the actual progress bar, as the larger numbers
makes people think that flatpak is more bloated than it actually is.

Another approach is to have an dynamic REST call where you give a list
of ref/commit/local_commit tuples and the server will compute (with
server-side caching) the list of download sizes. This would be
optional, but implemented by e.g. flat-manager and used by flatpak to
amend the download size shown in the table. This way we only do one
extra network roundtrip to get this info, and we only transfer the
information we need (the size) rather than all the details about all
the objects.

This call would be optional and if not implemented by the server, it
could just do whatever it currently does.