Re: Extending ostree's functionality (adding a metadat index)

[Vivek Dasmohapatra <vivek collabora co uk>]

> First, how about instead of "index" (which might confuse people who know
> git since it has nothing to do with git's "index"), we call it something
> like "cachedsizes"?  Or "sizelist"?  Below I'll call it "sizelist", but
> I'm happy to choose other names[1]

sizelist is fine with me.

> The object types are things that are "core" to the data model, where as
> this data is entirely derivative/duplicative; we're carrying it in the
> repo as an addition/optimization.  It should be possible to safely
> delete it and the only loss is that pull no longer gives you a reliable
> progress bar.  (Which is the current status quo)
>
> So I think that the .sizelist or whatever should simply be something
> that ostree-repo-pull.c knows about, but ostree-core.c maybe only has
> functions to retrieve the filename.

That's how this is currently implemented. The .sizelist is opportunistically
fetched if it is there and ignored if not: It isn't actually inserted
into the commit. If it's not there pull just carries on with its day
and doesn't complain.

> Concretely, now that libostree exists as a public shared library, I
> don't think we should expose the sizelist existence/internals there to
> potential consumers like PackageKit.

Ok, so make all the sizelist meta-meta-data private. I can do that.

> >   - said index is downloaded on pull if it is available
> >   - the index is entirely optional, pull doesn;t care if it isn't there

> Right.
>
> >   - pull has a metadata flag which pulls only the .commit and .index
>
> And then we're just printing it?  I'll look at your code, but I think
> what we want here is to cache the .sizelist for a given .commit, so we
> don't redownload it each time the upgrade UI queries it.

It's cached - we should only re-fetch if it's not here already.

> Given that what ot-builtin-summary.c is doing here is parsing
> the .sizelist only to add up the individual entries, how about we add an
> API like:
>
> gboolean
> ostree_repo_get_commit_disk_usage (OstreeRepo  *repo,
>                                   const char  *checksum,
>                                   guint64     *out_uncompressed_size,
>                                   guint64     *out_compressed_size,
>                                   GCancellable *cancellable,
>                                   GError     **error);
>
> Then ot-builtin-summary could do the first part of what it's doing now,
> with the partial metadata fetch.  Then it could simply invoke the above
> API, rather than parsing the internals of the sizelist itself.

Seems reasonable, I'll add this.

> >   - there are tests for the index creation and metadata pull, followed by
> >     a pull of the rest of the content
>
> Yay, tests!

> is obviously useful and I will want it for static deltas too.  However,
> can you match the existing commit style of:
>
> "topicname: What changed
>
> Why"

Sure, not a problem.

I'll ping the list again when the changes are ready.