Re: Idea: daily packs

[Owen Taylor <otaylor redhat com>]

On Mon, 2012-08-20 at 08:09 -0400, Colin Walters wrote:
> Hey Owen, thanks for looking at this!
> 
> On Fri, 2012-08-17 at 20:52 -0400, Owen Taylor wrote:
> 
> > The one part of this idea that I haven't figured out is how you would keep
> > the next day's 'ostree --pull' from downloading the next huge tarball instead
> > of a few thousand smaller files.
> 
> I wonder if a simple heuristic like "only use packfile if over 50% of
> the desired objects are in it" would work.  The pack indexes have this
> information now.  We could be smarter than that if we added the size of
> the objects to the index.

My thought was that that information wasn't available upfront - we don't
know the set of requested objects until we've done a large chunk of the
fetching.

So on a re-pull you'd start off with one object - the root tree - which
is in the new pack - but depending on where you go from there, you could
either end up downloading zillions of objects, or just 
/ /usr /usr/bin /usr/bin/zenity.

Is your thought that to start off fetching loose objects and at some
point decide to switch over to fetching the pack? It looks like the
trees are fetched depth-first before objects, so the switchover could be
pretty quick.

- Owen