Re: Moving existing ostree users to a new branch - take 2



On Fri, 2017-05-12 at 13:49 -0400, Colin Walters wrote:
On Fri, May 12, 2017, at 10:17 AM, Robert McQueen wrote:

As a side thought, we're a little worried about the summary file
getting
much larger over time as well. When you get up to 100s or 1000s of
Flatpaks we're talking a _lot_ of refs. A cool 10MB a day just to
refresh your app list (something you might very reasonably want to
do
daily, so you can implement some policy based off of that) isn't
too
great if that adds up to 300MB over a month - 60% of your data
plan.

Yeah...the summary is a very, very basic design.  I didn't put a lot
of
thought into it.  It kind of more came out of the requirements for
metalink <https://bugzilla.gnome.org/show_bug.cgi?id=729388>
which I never ended up actually using.

One thing I've thought about is basically making things recursive;
imagine we have a special ref:

ostree-metadata
whose contents are a filesystem tree that in turn contain whatever
data we want.  For example there might be a deltas/ subdirectory,
a refs/ subdirectory, etc.

So, I worry a bit too about summary file scalability, and whenever I
think about making it more complex I end up with this idea too, so I
think that is the correct way to do it long term. However, i'd like to
point out that summary-as-a-ref implies gpg signing the summary :)

However, I've also thought a bit about short-term fixes, as I add a
bunch of extra metadata to flatpak summaries that make them even
larger. For instance, the flathub summary is currently 286kb for 40
apps (and some themes) with related branches (like debuginfo and
locales) times 4 different arches.

Fortunately, it turns out that the summary files compress really well,
since they have a lot of repeating text. Just a simple gzip of the
flathub summary makes it go from 286k to 99k. 

We could easily get this win in transfer size automatically, in a
backwards compatatible way by using gzip transfer encoding for the
summary file. I think we just need to set the right HTTP headers when
we request the summary file and then make sure the server supports
compression.

I filed this issue about this: 
 https://github.com/ostreedev/ostree/issues/802

I would love if someone looked into this.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl redhat com            alexander larsson gmail com 
He's a sword-wielding small-town librarian gone bad. She's a manipulative 
nymphomaniac soap star from beyond the grave. They fight crime! 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]